terraform/website/upgrade-guides/0-13.html.markdown

485 lines
23 KiB
Markdown
Raw Normal View History

2020-06-03 02:22:46 +02:00
---
layout: "language"
2020-06-03 02:22:46 +02:00
page_title: "Upgrading to Terraform v0.13"
sidebar_current: "upgrade-guides-0-13"
description: |-
Upgrading to Terraform v0.13
---
# Upgrading to Terraform v0.13
Terraform v0.13 is a major release and thus includes some changes that
you'll need to consider when upgrading. This guide is intended to help with
that process.
The goal of this guide is to cover the most common upgrade concerns and
issues that would benefit from more explanation and background. The exhaustive
list of changes will always be the
[Terraform Changelog](https://github.com/hashicorp/terraform/blob/main/CHANGELOG.md).
2020-06-03 02:22:46 +02:00
After reviewing this guide, we recommend reviewing the Changelog to check for
specific notes about less-commonly-used features.
This guide focuses on changes from v0.12 to v0.13. Terraform supports upgrade
tools and features only for one major release upgrade at a time, so if you are
currently using a version of Terraform prior to v0.12 please upgrade through
the latest minor releases of all of the intermediate versions first, reviewing
the previous upgrade guides for any considerations that may be relevant to you.
In particular, Terraform v0.13 no longer includes the `terraform 0.12upgrade`
command for automatically migrating module source code from v0.11 to v0.12
syntax. If your modules are written for v0.11 and earlier you may need to
upgrade their syntax using the latest minor release of Terraform v0.12 before
using Terraform v0.13.
-> If you run into any problems during upgrading that are not addressed by the
information in this guide, please feel free to start a topic in
[The Terraform community forum](https://discuss.hashicorp.com/c/terraform-core),
describing the problem you've encountered in enough detail that other readers
may be able to reproduce it and offer advice.
Upgrade guide sections:
* [Before You Upgrade](#before-you-upgrade)
2020-06-03 02:22:46 +02:00
* [Explicit Provider Source Locations](#explicit-provider-source-locations)
* [New Filesystem Layout for Local Copies of Providers](#new-filesystem-layout-for-local-copies-of-providers)
* [Special considerations for in-house providers](#in-house-providers)
* [Destroy-time provisioners may not refer to other resources](#destroy-time-provisioners-may-not-refer-to-other-resources)
* [Data resource reads can no longer be disabled by `-refresh=false`](#data-resource-reads-can-no-longer-be-disabled-by-refresh-false)
* [Frequently Asked Questions](#frequently-asked-questions)
2020-06-03 02:22:46 +02:00
## Before You Upgrade
When upgrading between major releases, we always recommend ensuring that you
can run `terraform plan` and see no proposed changes on the previous version
first, because otherwise pending changes can add additional unknowns into the
upgrade process.
For this upgrade in particular, completing the upgrade will require running
`terraform apply` with Terraform 0.13 after upgrading in order to apply some
upgrades to the Terraform state, and we recommend doing that with no other
changes pending.
2020-06-03 02:22:46 +02:00
## Explicit Provider Source Locations
Prior versions of Terraform have supported automatic provider installation only
for providers packaged and distributed by HashiCorp. Providers built by the
community have previously required manual installation by extracting their
distribution packages into specific local filesystem locations.
Terraform v0.13 introduces a new hierarchical namespace for providers that
allows specifying both HashiCorp-maintained and community-maintained providers
as dependencies of a module, with community providers distributed from other
2021-11-04 21:19:35 +01:00
namespaces on [Terraform Registry](https://registry.terraform.io/) or from a
2020-06-03 02:22:46 +02:00
third-party provider registry.
In order to establish the hierarchical namespace, Terraform now requires
explicit source information for any providers that are not HashiCorp-maintained,
using a new syntax in the `required_providers` nested block inside the
`terraform` configuration block:
```hcl
terraform {
required_providers {
azurerm = {
# The "hashicorp" namespace is the new home for the HashiCorp-maintained
# provider plugins.
#
# source is not required for the hashicorp/* namespace as a measure of
# backward compatibility for commonly-used providers, but recommended for
# explicitness.
source = "hashicorp/azurerm"
version = "~> 2.12"
}
newrelic = {
# source is required for providers in other namespaces, to avoid ambiguity.
source = "newrelic/newrelic"
version = "~> 2.1.1"
}
}
}
```
If you are using providers that now require an explicit source location to be
specified, `terraform init` will produce an error like the following:
```
Error: Failed to install providers
Could not find required providers, but found possible alternatives:
hashicorp/datadog -> terraform-providers/datadog
hashicorp/fastly -> terraform-providers/fastly
If these suggestions look correct, upgrade your configuration with the
following command:
terraform 0.13upgrade
```
As mentioned in the error message, Terraform v0.13 includes an automatic
upgrade command
[`terraform 0.13upgrade`](/docs/cli/commands/0.13upgrade.html)
2020-06-03 02:22:46 +02:00
that is able to automatically generate source addresses for unlabelled
providers by consulting the same lookup table that was previously used for
Terraform v0.12 provider installation. This command will automatically modify
the configuration of your current module, so you can use the features of your
version control system to inspect the proposed changes before committing them.
We recommend running `terraform 0.13upgrade` even if you don't see the message,
because it will generate the recommended explicit source addresses for
providers in the "hashicorp" namespace.
For more information on declaring provider dependencies, see
[Provider Requirements](/docs/language/providers/requirements.html).
2020-06-03 02:22:46 +02:00
That page also includes some guidance on how to write provider dependencies
for a module that must remain compatible with both Terraform v0.12 and
Terraform v0.13; the `terraform 0.13upgrade` result includes a conservative
version constraint for Terraform v0.13 or later, which you can weaken to
`>= 0.12.26` if you follow the guidelines in
[v0.12-Compatible Provider Requirements](/docs/language/providers/requirements.html#v0-12-compatible-provider-requirements).
2020-06-03 02:22:46 +02:00
Each module must declare its own set of provider requirements, so if you have
a configuration which calls other modules then you'll need to run this upgrade
command for each module separately.
[The `terraform 0.13upgrade documentation`](/docs/cli/commands/0.13upgrade.html)
2020-06-03 02:22:46 +02:00
includes an example of running the upgrade process across all directories under
a particular prefix that contain `.tf` files using some common Unix command line
tools, which may be useful if you want to upgrade all modules in a single
repository at once.
After you've added explicit provider source addresses to your configuration,
run `terraform init` again to re-run the provider installer.
-> **Action:** Either run [`terraform 0.13upgrade`](/docs/cli/commands/0.13upgrade.html) for each of your modules, or manually update the provider declarations to use explicit source addresses.
2020-06-03 02:22:46 +02:00
The upgrade tool described above only updates references in your configuration.
The Terraform state also includes references to provider configurations which
need to be updated to refer to the correct providers.
Terraform will automatically update provider configuration references in the
state the first time you run `terraform apply` after upgrading, but it relies
on information in the configuration to understand which provider any
existing resource belongs to, and so you must run `terraform apply` at least
once (and accept any changes it proposes) before removing any `resource` blocks
from your configuration after upgrading.
If you are using Terraform Cloud or Terraform Enterprise with the VCS-driven
workflow (as opposed to CLI-driven runs), refer to
[The UI- and VCS-driven Run Workflow](/docs/cloud/run/ui.html) to learn how
to manually start a run after you select a Terraform v0.13 release for your
workspace.
If you remove a `resource` block (or a `module` block for a module that
contains `resource` blocks) before the first `terraform apply`, you may see
a message like this reflecting that Terraform cannot determine which provider
configuration the existing object ought to be managed by:
```
Error: Provider configuration not present
To work with null_resource.foo its original provider configuration at
provider["registry.terraform.io/-/aws"] is required, but it has been removed.
This occurs when a provider configuration is removed while objects created by
that provider still exist in the state. Re-add the provider configuration to
destroy aws_instance.example, after which you can remove the provider
configuration again.
```
In this specific upgrade situation the problem is actually the missing
`resource` block rather than the missing `provider` block: Terraform would
normally refer to the configuration to see if this resource has an explicit
`provider` argument that would override the default strategy for selecting
a provider. If you see the above after upgrading, re-add the resource mentioned
in the error message until you've completed the upgrade.
-> **Action:** After updating all modules in your configuration to use the new provider requirements syntax, run `terraform apply` to create a new state snapshot containing the new-style provider source addresses that are now specified in your configuration.
2020-06-03 02:22:46 +02:00
## New Filesystem Layout for Local Copies of Providers
As part of introducing the hierarchical provider namespace discussed in the
previous section, Terraform v0.13 also introduces a new hierarchical directory
structure for manually-installed providers in the local filesystem.
If you use local copies of official providers or if you use custom in-house
providers that you have installed manually, you will need to adjust your local
directories to use the new directory structure.
The previous layout was a single directory per target platform containing
various executable files named with the prefix `terraform-provider`, like
`linux_amd64/terraform-provider-google_v2.0.0`. The new expected location for the
Google Cloud Platform provider for that target platform within one of the local
search directories would be the following:
```
registry.terraform.io/hashicorp/google/2.0.0/linux_amd64/terraform-provider-google_v2.0.0
```
The `registry.terraform.io` above is the hostname of the registry considered
to be the origin for this provider. The provider source address
`hashicorp/google` is a shorthand for `registry.terraform.io/hashicorp/google`,
and the full, explicit form is required for a local directory.
Note that the version number given as a directory name must be written _without_
the "v" prefix that tends to be included when a version number is used as part
of a git branch name. If you include that prefix, Terraform will not recognize
the directory as containing provider packages.
As before, the recommended default location for locally-installed providers
is one of the following, depending on which operating system you are running
Terraform under:
* Windows: `%APPDATA%\terraform.d\plugins`
* All other systems: `~/.terraform.d/plugins`
Terraform v0.13 introduces some additional options for customizing where
Terraform looks for providers in the local filesystem. For more information on
those new options, see [Provider Installation](/docs/cli/config/config-file.html#provider-installation).
2020-06-03 02:22:46 +02:00
If you use only providers that are automatically installable from Terraform
provider registries but still want to avoid Terraform re-downloading them from
registries each time, Terraform v0.13 includes
[the `terraform providers mirror` command](/docs/cli/commands/providers/mirror.html)
2020-06-03 02:22:46 +02:00
which you can use to automatically populate a local directory based on the
requirements of the current configuration file:
```
terraform providers mirror ~/.terraform.d/plugins
```
-> **Action:** If you use local copies of official providers rather than installing them automatically from Terraform Registry, adopt the new expected directory structure for your local directory either by running `terraform providers mirror` or by manually reorganizing the existing files.
### In-house Providers
If you use an in-house provider that is not available from an upstream registry
at all, after upgrading you will see an error similar to the following:
```
- Finding latest version of hashicorp/happycloud...
Error: Failed to install provider
Error while installing hashicorp/happycloud: provider registry
registry.terraform.io does not have a provider named
registry.terraform.io/hashicorp/happycloud
```
Terraform assumes that a provider without an explicit source address belongs
to the "hashicorp" namespace on `registry.terraform.io`, which is not true
for your in-house provider. Instead, you can use any domain name under your
control to establish a _virtual_ source registry to serve as a separate
namespace for your local use. For example:
```
2020-06-25 19:17:16 +02:00
terraform.example.com/awesomecorp/happycloud/1.0.0/linux_amd64/terraform-provider-happycloud_v1.0.0
2020-06-03 02:22:46 +02:00
```
You can then specify explicitly the requirement for that in-house provider
in your modules, using the requirement syntax discussed in the previous section:
```hcl
terraform {
required_providers {
happycloud = {
source = "terraform.example.com/awesomecorp/happycloud"
version = "1.0.0"
}
}
}
```
If you wish, you can later run your own Terraform provider registry at the
specified hostname as an alternative to local installation, without any further
modifications to the above configuration. However, we recommend tackling that
only after your initial upgrade using the new local filesystem layout.
-> **Action:** If you use in-house providers that are not installable from a provider registry, assign them a new source address under a domain name you control and update your modules to specify that new source address.
If your configuration using one or more in-house providers has existing state
snapshots that include resources belonging to those providers, you'll also need
to perform a one-time migration of the provider references in the state, so
Terraform can understand them as belonging to your in-house providers rather
than to providers in the public Terraform Registry. If you are in this
situation, `terraform init` will produce the following error message after
you complete the configuration changes described above:
```
Error: Failed to install legacy providers required by state
2020-06-03 02:22:46 +02:00
Found unresolvable legacy provider references in state. It looks like these
refer to in-house providers. You can update the resources in state with the
following command:
terraform state replace-provider registry.terraform.io/-/happycloud terraform.example.com/awesomecorp/happycloud
2020-06-03 02:22:46 +02:00
```
Provider source addresses starting with `registry.terraform.io/-/` are a special
way Terraform marks legacy addresses where the true namespace is unknown.
For providers that were automatically-installable in Terraform 0.12, Terraform
0.13 can automatically determine the new addresses for these using a lookup
table in the public Terraform Registry, but for in-house providers you will
need to provide the appropriate mapping manually.
The `terraform state replace-provider` subcommand allows re-assigning provider
source addresses recorded in the Terraform state, and so we can use this
command to tell Terraform how to reinterpret the "legacy" provider addresses
as properly-namespaced providers that match with the provider source addresses
in the configuration.
~> **Warning:** The `terraform state replace-provider` subcommand, like all of the `terraform state` subcommands, will create a new state snapshot and write it to the configured backend. After the command succeeds the latest state snapshot will use syntax that Terraform v0.12 cannot understand, so you should perform this step only when you are ready to permanently upgrade to Terraform v0.13.
```
terraform state replace-provider 'registry.terraform.io/-/happycloud' 'terraform.example.com/awesomecorp/happycloud'
```
The command above asks Terraform to update any resource instance in the state
that belongs to a legacy (non-namespaced) provider called "happycloud" to
instead belong to the fully-qualified source address
`terraform.example.com/awesomecorp/happycloud`.
Whereas the configuration changes for provider requirements are made on a
per-module basis, the Terraform state captures data from throughout the
configuration (all of the existing module instances) and so you only need to
run `terraform state replace-provider` once per configuration.
Running `terraform init` again after completing this step should cause
Terraform to attempt to install `terraform.example.com/awesomecorp/happycloud`
and to find it in the local filesystem directory you populated in an earlier
step.
-> **Action:** If you use in-house providers that are not installable from a provider registry and your existing state contains resource instances that were created with any of those providers, use the `terraform state replace-provider` command to update the state to use the new source addressing scheme only once you are ready to commit to your v0.13 upgrade. (Terraform v0.12 cannot parse a state snapshot that was created by this command.)
## Destroy-time provisioners may not refer to other resources
Destroy-time provisioners allow introducing arbitrary additional actions into
the destroy phase of the resource lifecycle, but in practice the design of this
feature was flawed because it created the possibility for a destroy action
of one resource to depend on a create or update action of another resource,
which often leads either to dependency cycles or to incorrect behavior due to
unsuitable operation ordering.
In order to retain as many destroy-time provisioner capabilities as possible
while addressing those design flaws, Terraform v0.12.18 began reporting
deprecation warnings for any `provisioner` block setting `when = destroy` whose
configuration refers to any objects other than `self`, `count`, and `each`.
Addressing the flaws in the destroy-time provisioner design was a pre-requisite
for new features in v0.13 such as module `depends_on`, so Terraform v0.13
concludes the deprecation cycle by making such references now be fatal errors:
```
Error: Invalid reference from destroy provisioner
Destroy-time provisioners and their connection configurations may only
reference attributes of the related resource, via 'self', 'count.index',
or 'each.key'.
References to other resources during the destroy phase can cause dependency
cycles and interact poorly with create_before_destroy.
```
Some existing modules using resource or other references inside destroy-time
provisioners can be updated by placing the destroy-time provisioner inside a
`null_resource` resource and copying any data needed at destroy time into
the `triggers` map to be accessed via `self`:
```hcl
resource "null_resource" "example" {
triggers = {
instance_ip_addr = aws_instance.example.private_ip
}
provisioner "remote-exec" {
when = destroy
connection {
host = self.triggers.instance_ip_addr
# ...
}
# ...
}
}
```
In the above example, the `null_resource.example.triggers` map is effectively
acting as a temporary "cache" for the instance's private IP address to
guarantee that a value will be available when the provisioner runs, even if
the `aws_instance.example` object itself isn't currently available.
The provisioner's `connection` configuration can refer to that value via
`self`, whereas referring directly to `aws_instance.example.private_ip` in that
context is forbidden.
[Provisioners are a last resort](/docs/language/resources/provisioners/syntax.html#provisioners-are-a-last-resort),
2020-06-03 02:22:46 +02:00
so we recommend avoiding both create-time and destroy-time provisioners wherever
possible. Other options for destroy-time actions include using `systemd` to
run commands within your virtual machines during shutdown or using virtual
machine lifecycle hooks provided by your chosen cloud computing platform,
both of which can help ensure that the shutdown actions are taken even if the
virtual machine is terminated in an unusual way.
-> **Action:** If you encounter the "Invalid reference from destroy provisioner" error message after upgrading, reorganize your destroy-time provisioners to depend only on self-references, and consider other approaches if possible to avoid using destroy-time provisioners at all.
## Data resource reads can no longer be disabled by `-refresh=false`
In Terraform v0.12 and earlier, Terraform would read the data for data
resources during the "refresh" phase of `terraform plan`, which is the same
phase where Terraform synchronizes its state with any changes made to
remote objects.
An important prerequisite for properly supporting `depends_on` for both
data resources and modules containing data resources was to change the data
resource lifecycle to now read data during the _plan_ phase, so that
dependencies on managed resources could be properly respected.
If you were previously using `terraform plan -refresh=false` or
`terraform apply -refresh=false` to disable the refresh phase, you will find
that under Terraform 0.13 this will continue to disable synchronization of
managed resources (declared with `resource` blocks) but will no longer
disable the reading of data resources (declared with `data` blocks).
~> Updating the data associated with data resources is crucial to producing an
accurate plan, and so there is no replacement mechanism in Terraform v0.13
to restore the previous behavior.
## Frequently Asked Questions
### Why do I see `-/provider` during init?
Provider source addresses starting with `registry.terraform.io/-/` are a special
way Terraform marks legacy addresses where the true namespace is unknown. For
providers that were automatically-installable in Terraform 0.12, Terraform 0.13
can automatically determine the new addresses for these using a lookup table in
the public Terraform Registry. That lookup table is accessed by using the
special namespace `-`.
When you run `init`, terraform generates a list of required providers based on
both the configuration and state. Legacy-style providers - such as providers in
a statefile written with Terraform v0.12 - don't have a namespace, so terraform
uses the placeholder namespace `-` to query the registry. That is why you may
see output like this during your first `init`:
```
- Finding latest version of -/null...
- Finding latest version of -/random...
- Finding latest version of hashicorp/null...
- Finding latest version of hashicorp/random...
```
Terraform found providers `null` and `random` in the statefile without a
namespace. Terraform *also* found `hashicorp/null` and `hashicorp/random` in the
configuration files. Providers in configuration are automatically assumed to be
default (HashiCorp) providers, while providers found in state are first looked
up in the registry.
While this does not cause any problems for Terraform, it has been confusing. You
may circumvent this by using the `terraform state replace-provider` subcommand
to tell Terraform exactly what provider addresses are required in state.
Continuing from the example above, the following commands tell Terraform the
source address for the `null` and `random` providers:
```
terraform state replace-provider -- -/random registry.terraform.io/hashicorp/random
terraform state replace-provider -- -/null registry.terraform.io/hashicorp/null
```
If you are seeing these messages with errors, and are using in-house or
locally-installed providers, please see the section on [in-house providers](#in-house-providers).