Go to file
Martin Atkins 85eda8a059 state/remote: Don't persist snapshot for unchanged state
Previously we would write to the backend for every call to PersistState,
even if nothing changed since the last write, but update the serial only
if the state had changed.

The Terraform Cloud & Enterprise state storage have a simple safety check
that any future write with an already-used lineage and serial must be
byte-for-byte identical. StatesMarshalEqual is intended to detect that,
but it only actually detects changes the state itself, and not changes
to the snapshot metadata.

Because we write the current Terraform version into the snapshot metadata
during serialization, we'd previously have an issue where if the first
state write after upgrading Terraform to a new version happened to change
nothing about the state content then we'd write a new snapshot that
differed only by Terraform version, and Terraform Cloud/Enterprise would
then reject it.

The snapshot header is discarded immediately after decoding, so we can't
use information from it when deciding whether to increment the serial.
The next best thing is to skip sending no-op snapshot updates to the state
client in the first place.

These writes are unnecessary anyway, and state storage owners have asked
us in the past to elide these to avoid generating noise in their version
logs, so we'll also finally meet those requests as a nice side-effect of
this change.

We didn't previously have tests for the full flow of retrieving and then
successively updating persisted state snapshots, so this includes a test
which covers that logic and includes an assertion that a no-op update does
not get written to the state client.
2019-06-20 06:18:40 -07:00
.github build: use correct link for GCP provider in GitHub issue template 2019-04-18 14:06:24 -07:00
addrs stringer: Regenerate files with latest version 2019-05-13 15:34:27 +01:00
backend backend/remote-state: remove dead code 2019-06-18 14:18:05 -07:00
builtin add more timeout provider tests 2019-06-19 22:48:15 -04:00
command vendor: go get github.com/hashicorp/hcl2@0b64543c968c 2019-06-18 17:37:24 -07:00
communicator grammatical updates to comments and docs (#20195) 2019-03-21 14:05:41 -07:00
config Ensure matching modules with metadata when requested (#21640) 2019-06-07 14:29:05 -04:00
configs Merge pull request #21254 from davewongillies/gcs 2019-06-13 10:24:38 -04:00
contrib Remove support for the -module-depth flag 2018-11-02 18:44:04 +01:00
dag terraform: ugly huge change to weave in new HCL2-oriented types 2018-10-16 18:46:46 -07:00
digraph Fix TestWriteDot random order error 2014-07-29 10:26:50 -07:00
docs docs: README for the plugin-protocol directory 2019-06-11 14:29:35 -07:00
e2e [WIP] Re-enable the end-to-end tests (#20044) 2019-04-29 13:03:24 -04:00
examples Fix Google Cloud Platform name across docs. 2019-01-15 12:10:20 -08:00
flatmap flatmap: be resilient to lying "foo.#" key 2017-06-23 14:47:36 -07:00
helper private and timeout handling in grpc_provider 2019-06-19 22:48:15 -04:00
httpclient Allow callers to append to user agent 2018-03-15 10:53:44 -04:00
internal Merge pull request #21254 from davewongillies/gcs 2019-06-13 10:24:38 -04:00
lang Don't allow nulls in calls to merge 2019-06-14 15:26:24 -04:00
moduledeps plugin/discovery: PluginRequirements can specify SHA256 digests 2017-06-09 14:03:59 -07:00
plans stringer: Regenerate files with latest version 2019-05-13 15:34:27 +01:00
plugin send and receive Private through ReadResource 2019-06-03 18:08:26 -04:00
providers send and receive Private through ReadResource 2019-06-03 18:08:26 -04:00
provisioners provisioners: Add Factory type and FactoryFixed helper 2018-10-16 19:14:11 -07:00
registry Add friendly error for when registry unresponsive 2019-03-27 14:39:14 -04:00
repl command: "terraform init" can partially initialize for 0.12upgrade 2019-01-14 11:33:21 -08:00
scripts Create smaller docker images for releases 2019-04-10 10:06:04 -07:00
state state/remote: Don't persist snapshot for unchanged state 2019-06-20 06:18:40 -07:00
states Merge pull request #21484 from minamijoyo/fix-statev2-hash-decode 2019-06-10 17:52:33 -04:00
svchost backend/remote: cleanup test connections 2019-02-07 09:55:19 +01:00
terraform website: Document ignore_changes for individual map elements 2019-06-18 17:37:24 -07:00
test-fixtures main: allow overriding host-based discovery in CLI config 2017-10-26 08:58:52 -07:00
tfdiags stringer: Regenerate files with latest version 2019-05-13 15:34:27 +01:00
tools plugin/discovery: Return tfdiags from Get 2019-03-18 12:21:27 -04:00
vendor vendor: go get github.com/hashicorp/hcl2@0b64543c968c 2019-06-18 17:37:24 -07:00
version release: clean up after v0.12.2 2019-06-12 20:22:18 +00:00
website website: Clarifications to Puppet provisioner docs (#21810) 2019-06-20 09:15:30 -04:00
.gitignore gitignore should ignore test files that use .terraform/tfstate 2017-01-26 14:33:49 -08:00
.go-version go-version: Bump to 1.12.4 2019-05-13 15:34:21 +01:00
.hashibot.hcl fix slack channel ref 2019-06-17 23:11:53 -04:00
.tfdev Mildwonkey/upgrade doc remote backend (#21201) 2019-05-03 15:34:54 -04:00
.travis.yml travis: Bump go to 1.12.4 2019-05-13 15:55:43 +01:00
BUILDING.md Makefile/docs: Lock in 1.6 req, doc vendored deps 2016-02-24 16:13:49 -06:00
CHANGELOG.md Fix broken markdown link in CHANGELOG.md (#21767) 2019-06-18 09:53:56 -04:00
CODEOWNERS First pass at adding CODEOWNERS to link remote-state backends with maintainers of the associated providers. 2019-02-11 15:52:19 -08:00
Dockerfile build: Stop using deprecated MAINTAINER in Dockerfile 2017-10-27 17:25:44 -07:00
LICENSE Adding license 2014-07-28 13:54:06 -04:00
Makefile build: Run "go generate" in modules mode 2019-02-06 11:19:44 -08:00
README.md README: Fix typo 2019-03-12 10:40:45 -07:00
checkpoint.go fixing version numbers RCs should be labeled x.x.x-rcx 2015-02-07 16:56:56 +01:00
commands.go command/jsonprovider: export providers schemas to json (#20446) 2019-02-25 13:32:47 -08:00
config.go main: allow overriding host-based discovery in CLI config 2017-10-26 08:58:52 -07:00
config_test.go main: allow overriding host-based discovery in CLI config 2017-10-26 08:58:52 -07:00
config_unix.go Use build-in method to get user homedir instead of eval on sh 2018-03-21 14:55:56 +01:00
config_windows.go config looks in a plugin directory if it exists 2014-09-27 12:36:13 -07:00
go.mod vendor: go get github.com/hashicorp/hcl2@0b64543c968c 2019-06-18 17:37:24 -07:00
go.sum vendor: go get github.com/hashicorp/hcl2@0b64543c968c 2019-06-18 17:37:24 -07:00
help.go help: Make version and help flags consistent 2018-08-01 14:28:39 -07:00
main.go Adding documentation for TF_CLI_CONFIG_FILE environment variable (#20834) 2019-04-05 14:21:40 -04:00
main_test.go main: make configuration available when initializing commands 2017-09-29 14:03:09 -07:00
panic.go panic: Instruct the user to include terraform's version for bug reports. 2015-05-14 18:14:56 -04:00
plugins.go keep .terraform.d/plugins for discovery 2017-08-09 17:46:49 -04:00
signal_unix.go Forward SIGTERM and handle that as an interrupt 2016-12-08 12:20:25 -05:00
signal_windows.go Forward SIGTERM and handle that as an interrupt 2016-12-08 12:20:25 -05:00
synchronized_writers.go main: synchronize writes to VT100-faker on Windows 2017-05-04 15:36:51 -07:00
version.go states/statemgr: Fix the Filesystem state manager tests 2018-11-19 09:02:35 -08:00

README.md

Terraform

Terraform

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

The key features of Terraform are:

  • Infrastructure as Code: Infrastructure is described using a high-level configuration syntax. This allows a blueprint of your datacenter to be versioned and treated as you would any other code. Additionally, infrastructure can be shared and re-used.

  • Execution Plans: Terraform has a "planning" step where it generates an execution plan. The execution plan shows what Terraform will do when you call apply. This lets you avoid any surprises when Terraform manipulates infrastructure.

  • Resource Graph: Terraform builds a graph of all your resources, and parallelizes the creation and modification of any non-dependent resources. Because of this, Terraform builds infrastructure as efficiently as possible, and operators get insight into dependencies in their infrastructure.

  • Change Automation: Complex changesets can be applied to your infrastructure with minimal human interaction. With the previously mentioned execution plan and resource graph, you know exactly what Terraform will change and in what order, avoiding many possible human errors.

For more information, see the introduction section of the Terraform website.

Getting Started & Documentation

If you're new to Terraform and want to get started creating infrastructure, please checkout our Getting Started guide, available on the Terraform website.

All documentation is available on the Terraform website:

Developing Terraform

If you wish to work on Terraform itself or any of its built-in providers, you'll first need Go installed on your machine (version 1.11+ is required). Alternatively, you can use the Vagrantfile in the root of this repo to stand up a virtual machine with the appropriate dev tooling already set up for you.

This repository contains only Terraform core, which includes the command line interface and the main graph engine. Providers are implemented as plugins that each have their own repository in the terraform-providers organization on GitHub. Instructions for developing each provider are in the associated README file. For more information, see the provider development overview.

For local development of Terraform core, first make sure Go is properly installed and that a GOPATH has been set. You will also need to add $GOPATH/bin to your $PATH.

Next, using Git, clone this repository into $GOPATH/src/github.com/hashicorp/terraform.

You'll need to run make tools to install some required tools, then make. This will compile the code and then run the tests. If this exits with exit status 0, then everything is working! You only need to run make tools once (or when the tools change).

$ cd "$GOPATH/src/github.com/hashicorp/terraform"
$ make tools
$ make

To compile a development version of Terraform and the built-in plugins, run make dev. This will build everything using gox and put Terraform binaries in the bin and $GOPATH/bin folders:

$ make dev
...
$ bin/terraform
...

If you're developing a specific package, you can run tests for just that package by specifying the TEST variable. For example below, onlyterraform package tests will be run.

$ make test TEST=./terraform
...

If you're working on a specific provider which has not been separated into an individual repository and only wish to rebuild that provider, you can use the plugin-dev target. For example, to build only the Test provider:

$ make plugin-dev PLUGIN=provider-test

Dependencies

Terraform uses Go Modules for dependency management, but for the moment is continuing to use Go 1.6-style vendoring for compatibility with tools that have not yet been updated for full Go Modules support.

If you're developing Terraform, there are a few tasks you might need to perform.

Adding a dependency

If you're adding a dependency, you'll need to vendor it in the same Pull Request as the code that depends on it. You should do this in a separate commit from your code, as makes PR review easier and Git history simpler to read in the future.

To add a dependency:

Assuming your work is on a branch called my-feature-branch, the steps look like this:

  1. Add an import statement to a suitable package in the Terraform code.

  2. Run go mod vendor to download the latest version of the module containing the imported package into the vendor/ directory, and update the go.mod and go.sum files.

  3. Review the changes in git and commit them.

Updating a dependency

To update a dependency:

  1. Run go get -u module-path@version-number, such as go get -u github.com/hashicorp/hcl@2.0.0

  2. Run go mod vendor to update the vendored copy in the vendor/ directory.

  3. Review the changes in git and commit them.

Acceptance Tests

Terraform has a comprehensive acceptance test suite covering the built-in providers. Our Contributing Guide includes details about how and when to write and run acceptance tests in order to help contributions get accepted quickly.

Cross Compilation and Building for Distribution

If you wish to cross-compile Terraform for another architecture, you can set the XC_OS and XC_ARCH environment variables to values representing the target operating system and architecture before calling make. The output is placed in the pkg subdirectory tree both expanded in a directory representing the OS/architecture combination and as a ZIP archive.

For example, to compile 64-bit Linux binaries on Mac OS X, you can run:

$ XC_OS=linux XC_ARCH=amd64 make bin
...
$ file pkg/linux_amd64/terraform
terraform: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped

XC_OS and XC_ARCH can be space separated lists representing different combinations of operating system and architecture. For example, to compile for both Linux and Mac OS X, targeting both 32- and 64-bit architectures, you can run:

$ XC_OS="linux darwin" XC_ARCH="386 amd64" make bin
...
$ tree ./pkg/ -P "terraform|*.zip"
./pkg/
├── darwin_386
│   └── terraform
├── darwin_386.zip
├── darwin_amd64
│   └── terraform
├── darwin_amd64.zip
├── linux_386
│   └── terraform
├── linux_386.zip
├── linux_amd64
│   └── terraform
└── linux_amd64.zip

4 directories, 8 files

Note: Cross-compilation uses gox, which requires toolchains to be built with versions of Go prior to 1.5. In order to successfully cross-compile with older versions of Go, you will need to run gox -build-toolchain before running the commands detailed above.

Docker

When using docker you don't need to have any of the Go development tools installed and you can clone terraform to any location on disk (doesn't have to be in your $GOPATH). This is useful for users who want to build master or a specific branch for testing without setting up a proper Go environment.

For example, run the following command to build terraform in a linux-based container for macOS.

docker run --rm -v $(pwd):/go/src/github.com/hashicorp/terraform -w /go/src/github.com/hashicorp/terraform -e XC_OS=darwin -e XC_ARCH=amd64 golang:latest bash -c "apt-get update && apt-get install -y zip && make bin"

License

FOSSA Status