diff --git a/tools/terraform-bundle/README.md b/tools/terraform-bundle/README.md index b95a65ee8..5d6bf103b 100644 --- a/tools/terraform-bundle/README.md +++ b/tools/terraform-bundle/README.md @@ -9,6 +9,10 @@ work with a particular configuration, but sometimes Terraform is deployed in a network that, for one reason or another, cannot access the official plugin repository for automatic download. +In some cases, this can be solved by installing provider plugins into the +[user plugins directory](https://www.terraform.io/docs/configuration/providers.html#third-party-plugins). +However, this doesn't always meet the needs of automated deployments. + `terraform-bundle` provides an alternative, by allowing the auto-download process to be run out-of-band on a separate machine that _does_ have access to the repository. The result is a zip file that can be extracted onto the @@ -54,7 +58,7 @@ providers { # the bundle archive. google = ["~> 1.0", "~> 2.0"] - # Include a custom plugin to the bundle. Will search for the plugin in the + # Include a custom plugin to the bundle. Will search for the plugin in the # plugins directory, and package it with the bundle archive. Plugin must have # a name of the form: terraform-provider-*, and must be build with the operating # system and architecture that terraform enterprise is running, e.g. linux and amd64 @@ -107,11 +111,12 @@ distinguished from official release archives and from each other when multiple bundles contain the same core Terraform version. To include custom plugins in the bundle file, create a local directory "./plugins" -and put all the plugins you want to include there. Optionally, you can use the +and put all the plugins you want to include there. Optionally, you can use the `-plugin-dir` flag to specify a location where to find the plugins. To be recognized -as a valid plugin, the file must have a name of the form: "terraform-provider-*-v*". In -addition, ensure that the plugin is build using the same operating system and -architecture used for terraform enterprise. Typically this will be linux and amd64. +as a valid plugin, the file must have a name of the form +`terraform-provider--v`. In +addition, ensure that the plugin is built using the same operating system and +architecture used for Terraform Enterprise. Typically this will be `linux` and `amd64`. ## Provider Resolution Behavior @@ -123,7 +128,12 @@ Therefore if automatic installation is not desired, it is important to ensure that version constraints within Terraform configurations do not exclude all of the versions available from the bundle. If a suitable version cannot be found in the bundle, Terraform _will_ attempt to satisfy that dependency by -automatic installation from the official repository. +automatic installation from the official repository. If you want +`terraform init` to explicitly fail instead of contacting the repository, pass +the `-get-plugins=false` option. + +For full details about provider resolution, see +[How Terraform Works: Plugin Discovery](https://www.terraform.io/docs/extend/how-terraform-works.html#discovery). The downloaded provider archives are verified using the same signature check that is used for auto-installed plugins, using Hashicorp's release key. At @@ -133,21 +143,22 @@ that may change in a future version of this tool. ## Installing a Bundle in On-premises Terraform Enterprise If using a private install of Terraform Enterprise in an "air-gapped" -environment, this tool can produce a custom _tool package_ for Terraform, which +environment, this tool can produce a custom Terraform version package, which includes a set of provider plugins along with core Terraform. To create a suitable bundle, use the `-os` and `-arch` options as described above to produce a bundle targeting `linux_amd64`. You can then place this archive on an HTTP server reachable by the Terraform Enterprise hosts and install it as per -[Managing Tool Versions](https://github.com/hashicorp/terraform-enterprise-modules/blob/master/docs/managing-tool-versions.md). +[Administration: Managing Terraform Versions](https://www.terraform.io/docs/enterprise/private/admin/resources.html#managing-terraform-versions). -After choosing the "Add Tool Version" button, be sure to set the Tool to -"terraform" and then enter as the Version the generated bundle version from -the bundle filename, which will be of the form `N.N.N-bundleYYYYMMDDHH`. -Enter the URL at which the generated bundle archive can be found, and the -SHA256 hash of the file which can be determined by running the tool -`sha256sum` with the given file. +After clicking the "Add Terraform Version" button: + +1. In the "Version" field, enter the generated bundle version from the bundle + filename, which will be of the form `N.N.N-bundleYYYYMMDDHH`. +2. In the "URL" field, enter the URL where the generated bundle archive can be found. +3. In the "SHA256 Checksum" field, enter the SHA256 hash of the file, which can + be found by running `sha256sum ` or `shasum -a256 `. The new bundle version can then be selected as the Terraform version for any workspace. When selected, configurations that require only plugins diff --git a/website/docs/commands/init.html.markdown b/website/docs/commands/init.html.markdown index 87e708c5c..1c78ce7a0 100644 --- a/website/docs/commands/init.html.markdown +++ b/website/docs/commands/init.html.markdown @@ -116,37 +116,42 @@ initialized with its child modules. ## Plugin Installation -During init, the configuration is searched for both direct and indirect -references to [providers](/docs/configuration/providers.html), and the plugins -for the providers are retrieved from the plugin repository. The downloaded -plugins are installed to a subdirectory of the working directory, and are thus -local to that working directory. +During init, Terraform searches the configuration for both direct and indirect +references to providers and attempts to load the required plugins. -Re-running init with plugins already installed will install plugins only for -any providers that were added to the configuration since the last init. Use -`-upgrade` to additionally update already-installed plugins to the latest -versions that comply with the version constraints given in configuration. +For [providers distributed by HashiCorp](/docs/providers/index.html), +init will automatically download and install plugins if necessary. Plugins +can also be manually installed in the user plugins directory, located at +`~/.terraform.d/plugins` on most operating systems and +`\plugins` on Windows. -To skip plugin installation, use `-get-plugins=false`. +For more information about configuring and installing providers, see +[Configuration: Providers](/docs/configuration/providers.html). -The automatic plugin installation behavior can be overridden by extracting -the desired providers into a local directory and using the additional option -`-plugin-dir=PATH`. When this option is specified, _only_ the given directory -is consulted, which prevents Terraform from making requests to the plugin -repository or looking for plugins in other local directories. Passing an empty -string to `-plugin-dir` removes any previously recorded paths. +On subsequent runs, init only installs providers without acceptable versions +installed. (This includes newly added providers, and providers whose installed +versions can't meet the current version constraints.) Use `-upgrade` if you want +to update _all_ providers to the newest acceptable version. -Custom plugins can be used along with automatically installed plugins by -placing them in `terraform.d/plugins/OS_ARCH/` inside the directory being -initialized. Plugins found here will take precedence if they meet the required -constraints in the configuration. The `init` command will continue to -automatically download other plugins as needed. +You can modify `terraform init`'s plugin behavior with the following options: -When plugins are automatically downloaded and installed, by default the -contents are verified against an official HashiCorp release signature to -ensure that they were not corrupted or tampered with during download. It is -recommended to allow Terraform to make these checks, but if desired they may -be disabled using the option `-verify-plugins=false`. +- `-upgrade` — Update all previously installed plugins to the newest version + that complies with the configuration's version constraints. This option does + not apply to manually installed plugins. +- `-get-plugins=false` — Skips plugin installation. Terraform will use plugins + installed in the user plugins directory, and any plugins already installed + for the current working directory. If the installed plugins aren't sufficient + for the configuration, init fails. +- `-plugin-dir=PATH` — Skips plugin installation and loads plugins _only_ from + the specified directory. This ignores the user plugins directory and any + plugins already installed in the current working directory. To restore the + default behavior after using this option, run init again and pass an empty + string to `-plugin-dir`. +- `-verify-plugins=false` — Skips release signature validation when + installing downloaded plugins (not recommended). Official plugin releases are + digitally signed by HashiCorp, and Terraform verifies these signatures when + automatically downloading plugins. This option disables that verification. + (Terraform does not check signatures for manually installed plugins.) ## Running `terraform init` in automation diff --git a/website/docs/configuration/providers.html.md b/website/docs/configuration/providers.html.md index 12a717ef0..2a4154da3 100644 --- a/website/docs/configuration/providers.html.md +++ b/website/docs/configuration/providers.html.md @@ -66,6 +66,14 @@ Provider initialization is one of the actions of `terraform init`. Running this command will download and initialize any providers that are not already initialized. +Providers downloaded by `terraform init` are only installed for the current +working directory; other working directories can have their own installed +provider versions. + +Note that `terraform init` cannot automatically download providers that are not +distributed by HashiCorp. See [Third-party Plugins](#third-party-plugins) below +for installation instructions. + For more information, see [the `terraform init` command](/docs/commands/init.html). @@ -207,24 +215,67 @@ may also be used, but currently may cause errors when running `terraform destroy ## Third-party Plugins -At present Terraform can automatically install only the providers distributed -by HashiCorp. Third-party providers can be manually installed by placing -their plugin executables in one of the following locations depending on the -host operating system: +Anyone can develop and distribute their own Terraform providers. (See +[Writing Custom Providers](/docs/extend/writing-custom-providers.html) for more +about provider development.) These third-party providers must be manually +installed, since `terraform init` cannot automatically download them. -* On Windows, in the sub-path `terraform.d/plugins` beneath your user's - "Application Data" directory. -* On all other systems, in the sub-path `.terraform.d/plugins` in your - user's home directory. +Install third-party providers by placing their plugin executables in the user +plugins directory. The user plugins directory is in one of the following +locations, depending on the host operating system: -`terraform init` will search this directory for additional plugins during -plugin initialization. +Operating system | User plugins directory +------------------|----------------------- +Windows | `terraform.d\plugins` in your user's "Application Data" directory +All other systems | `.terraform.d/plugins` in your user's home directory -The naming scheme for provider plugins is `terraform-provider-NAME_vX.Y.Z`, +Once a plugin is installed, `terraform init` can initialize it normally. + +Providers distributed by HashiCorp can also go in the user plugins directory. If +a manually installed version meets the configuration's version constraints, +Terraform will use it instead of downloading that provider. This is useful in +airgapped environments and when testing pre-release provider builds. + +### Plugin Names and Versions + +The naming scheme for provider plugins is `terraform-provider-_vX.Y.Z`, and Terraform uses the name to understand the name and version of a particular -provider binary. Third-party plugins will often be distributed with an -appropriate filename already set in the distribution archive so that it can -be extracted directly into the plugin directory described above. +provider binary. + +If multiple versions of a plugin are installed, Terraform will use the newest +version that meets the configuration's version constraints. + +Third-party plugins are often distributed with an appropriate filename already +set in the distribution archive, so that they can be extracted directly into the +user plugins directory. + +### OS and Architecture Directories + +Terraform plugins are compiled for a specific operating system and architecture, +and any plugins in the root of the user plugins directory must be compiled for +the current system. + +If you use the same plugins directory on multiple systems, you can install +plugins into subdirectories with a naming scheme of `_` (for example, +`darwin_amd64`). Terraform uses plugins from the root of the plugins directory +and from the subdirectory that corresponds to the current system, ignoring +other subdirectories. + +Terraform's OS and architecture strings are the standard ones used by the Go +language. The following are the most common: + +* `darwin_amd64` +* `freebsd_386` +* `freebsd_amd64` +* `freebsd_arm` +* `linux_386` +* `linux_amd64` +* `linux_arm` +* `openbsd_386` +* `openbsd_amd64` +* `solaris_amd64` +* `windows_386` +* `windows_amd64` ## Provider Plugin Cache diff --git a/website/docs/plugins/basics.html.md b/website/docs/plugins/basics.html.md index c3cc738df..dafcec5bb 100644 --- a/website/docs/plugins/basics.html.md +++ b/website/docs/plugins/basics.html.md @@ -10,7 +10,7 @@ description: |- ~> **Advanced topic!** Plugin development is a highly advanced topic in Terraform, and is not required knowledge for day-to-day usage. -If you don't plan on writing any plugins, this section of the documentation is +If you don't plan on writing any plugins, this section of the documentation is not necessary to read. For general use of Terraform, please see our [Intro to Terraform](/intro/index.html) and [Getting Started](/intro/getting-started/install.html) guides. @@ -34,15 +34,22 @@ The network communication and RPC is handled automatically by higher-level Terraform libraries. The exact interface to implement is documented in its respective documentation section. -## Installing a Plugin +## Installing Plugins -To install a plugin distributed by a third party developer, place the binary -(extracted from any containing zip file) in -[the third-party plugins directory](/docs/configuration/providers.html#third-party-plugins). +The [provider plugins distributed by HashiCorp](/docs/providers/index.html) are +automatically installed by `terraform init`. Third-party plugins (both +providers and provisioners) can be manually installed into the user plugins +directory, located at `\terraform.d\plugins` on Windows and +`~/.terraform.d/plugins` on other systems. -Provider plugin binaries are named with the prefix `terraform-provider-`, -while provisioner plugins have the prefix `terraform-provisioner-`. Both -are placed in the same directory. +For more information, see: + +- [Configuring Providers](/docs/configuration/providers.html) +- [Configuring Providers: Third-party Plugins](/docs/configuration/providers.html#third-party-plugins) + +For developer-centric documentation, see: + +- [How Terraform Works: Plugin Discovery](/docs/extend/how-terraform-works.html#discovery) ## Developing a Plugin diff --git a/website/guides/running-terraform-in-automation.html.md b/website/guides/running-terraform-in-automation.html.md index 8987f25db..752c76117 100644 --- a/website/guides/running-terraform-in-automation.html.md +++ b/website/guides/running-terraform-in-automation.html.md @@ -156,7 +156,7 @@ applied, as opposed to some later plan that also exists. Different orchestration tools address this in different ways, but generally this is implemented via a _build pipeline_ feature, where different steps can be applied in sequence, with later steps having access to data produced -by earlier steps. +by earlier steps. The recommended approach is to allow only one plan to be outstanding at a time. When a plan is applied, any other existing plans that were produced @@ -285,14 +285,14 @@ $ ls -lah /usr/lib/custom-terraform-plugins ``` The version information at the end of the filenames is important so that -Terraform can infer the version number of each plugin. It is allowed to -concurrently install multiple versions of the same provider plugin, -which will then be used to satisfy +Terraform can infer the version number of each plugin. Multiple versions of the +same provider plugin can be installed, and Terraform will use the newest one +that matches the [provider version constraints](/docs/configuration/providers.html#provider-versions) -from Terraform configurations. +in the Terraform configuration. With this directory populated, the usual auto-download and -[plugin discovery](/docs/plugins/basics.html#installing-a-plugin) +[plugin discovery](/docs/extend/how-terraform-works.html#discovery) behavior can be bypassed using the `-plugin-dir` option to `terraform init`: * `terraform init -input=false -plugin-dir=/usr/lib/custom-terraform-plugins` @@ -307,7 +307,7 @@ unique constraints within each organization. Plugins can also be provided along with the configuration by creating a `terraform.d/plugins/OS_ARCH` directory, which will be searched before automatically downloading additional plugins. The `-get-plugins=false` flag can -be used to prevent Terraform from automatically downloading additional plugins. +be used to prevent Terraform from automatically downloading additional plugins. ## Terraform Enterprise