432 lines
20 KiB
Markdown
432 lines
20 KiB
Markdown
---
|
|
layout: "language"
|
|
page_title: "Backend Type: s3"
|
|
sidebar_current: "docs-backends-types-standard-s3"
|
|
description: |-
|
|
Terraform can store state remotely in S3 and lock that state with DynamoDB.
|
|
---
|
|
|
|
# S3
|
|
|
|
**Kind: Standard (with locking via DynamoDB)**
|
|
|
|
Stores the state as a given key in a given bucket on
|
|
[Amazon S3](https://aws.amazon.com/s3/).
|
|
This backend also supports state locking and consistency checking via
|
|
[Dynamo DB](https://aws.amazon.com/dynamodb/), which can be enabled by setting
|
|
the `dynamodb_table` field to an existing DynamoDB table name.
|
|
A single DynamoDB table can be used to lock multiple remote state files. Terraform generates key names that include the values of the `bucket` and `key` variables.
|
|
|
|
~> **Warning!** It is highly recommended that you enable
|
|
[Bucket Versioning](http://docs.aws.amazon.com/AmazonS3/latest/UG/enable-bucket-versioning.html)
|
|
on the S3 bucket to allow for state recovery in the case of accidental deletions and human error.
|
|
|
|
## Example Configuration
|
|
|
|
```hcl
|
|
terraform {
|
|
backend "s3" {
|
|
bucket = "mybucket"
|
|
key = "path/to/my/key"
|
|
region = "us-east-1"
|
|
}
|
|
}
|
|
```
|
|
|
|
This assumes we have a bucket created called `mybucket`. The
|
|
Terraform state is written to the key `path/to/my/key`.
|
|
|
|
Note that for the access credentials we recommend using a
|
|
[partial configuration](/docs/language/settings/backends/configuration.html#partial-configuration).
|
|
|
|
### S3 Bucket Permissions
|
|
|
|
Terraform will need the following AWS IAM permissions on
|
|
the target backend bucket:
|
|
|
|
* `s3:ListBucket` on `arn:aws:s3:::mybucket`
|
|
* `s3:GetObject` on `arn:aws:s3:::mybucket/path/to/my/key`
|
|
* `s3:PutObject` on `arn:aws:s3:::mybucket/path/to/my/key`
|
|
* `s3:DeleteObject` on `arn:aws:s3:::mybucket/path/to/my/key`
|
|
|
|
This is seen in the following AWS IAM Statement:
|
|
|
|
```json
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": "s3:ListBucket",
|
|
"Resource": "arn:aws:s3:::mybucket"
|
|
},
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
|
|
"Resource": "arn:aws:s3:::mybucket/path/to/my/key"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
-> **Note:** AWS can control access to S3 buckets with either IAM policies
|
|
attached to users/groups/roles (like the example above) or resource policies
|
|
attached to bucket objects (which look similar but also require a `Principal` to
|
|
indicate which entity has those permissions). For more details, see Amazon's
|
|
documentation about
|
|
[S3 access control](https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html).
|
|
|
|
### DynamoDB Table Permissions
|
|
|
|
If you are using state locking, Terraform will need the following AWS IAM
|
|
permissions on the DynamoDB table (`arn:aws:dynamodb:::table/mytable`):
|
|
|
|
* `dynamodb:GetItem`
|
|
* `dynamodb:PutItem`
|
|
* `dynamodb:DeleteItem`
|
|
|
|
This is seen in the following AWS IAM Statement:
|
|
|
|
```json
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": [
|
|
"dynamodb:GetItem",
|
|
"dynamodb:PutItem",
|
|
"dynamodb:DeleteItem"
|
|
],
|
|
"Resource": "arn:aws:dynamodb:*:*:table/mytable"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Data Source Configuration
|
|
|
|
To make use of the S3 remote state in another configuration, use the
|
|
[`terraform_remote_state` data
|
|
source](/docs/language/state/remote-state-data.html).
|
|
|
|
```hcl
|
|
data "terraform_remote_state" "network" {
|
|
backend = "s3"
|
|
config = {
|
|
bucket = "terraform-state-prod"
|
|
key = "network/terraform.tfstate"
|
|
region = "us-east-1"
|
|
}
|
|
}
|
|
```
|
|
|
|
The `terraform_remote_state` data source will return all of the root module
|
|
outputs defined in the referenced remote state (but not any outputs from
|
|
nested modules unless they are explicitly output again in the root). An
|
|
example output might look like:
|
|
|
|
```
|
|
data.terraform_remote_state.network:
|
|
id = 2016-10-29 01:57:59.780010914 +0000 UTC
|
|
addresses.# = 2
|
|
addresses.0 = 52.207.220.222
|
|
addresses.1 = 54.196.78.166
|
|
backend = s3
|
|
config.% = 3
|
|
config.bucket = terraform-state-prod
|
|
config.key = network/terraform.tfstate
|
|
config.region = us-east-1
|
|
elb_address = web-elb-790251200.us-east-1.elb.amazonaws.com
|
|
public_subnet_id = subnet-1e05dd33
|
|
```
|
|
|
|
## Configuration
|
|
|
|
This backend requires the configuration of the AWS Region and S3 state storage. Other configuration, such as enabling DynamoDB state locking, is optional.
|
|
|
|
### Credentials and Shared Configuration
|
|
|
|
The following configuration is required:
|
|
|
|
* `region` - (Required) AWS Region of the S3 Bucket and DynamoDB Table (if used). This can also be sourced from the `AWS_DEFAULT_REGION` and `AWS_REGION` environment variables.
|
|
|
|
The following configuration is optional:
|
|
|
|
* `access_key` - (Optional) AWS access key. If configured, must also configure `secret_key`. This can also be sourced from the `AWS_ACCESS_KEY_ID` environment variable, AWS shared credentials file (e.g. `~/.aws/credentials`), or AWS shared configuration file (e.g. `~/.aws/config`).
|
|
* `secret_key` - (Optional) AWS access key. If configured, must also configure `access_key`. This can also be sourced from the `AWS_SECRET_ACCESS_KEY` environment variable, AWS shared credentials file (e.g. `~/.aws/credentials`), or AWS shared configuration file (e.g. `~/.aws/config`).
|
|
* `iam_endpoint` - (Optional) Custom endpoint for the AWS Identity and Access Management (IAM) API. This can also be sourced from the `AWS_IAM_ENDPOINT` environment variable.
|
|
* `max_retries` - (Optional) The maximum number of times an AWS API request is retried on retryable failure. Defaults to 5.
|
|
* `profile` - (Optional) Name of AWS profile in AWS shared credentials file (e.g. `~/.aws/credentials`) or AWS shared configuration file (e.g. `~/.aws/config`) to use for credentials and/or configuration. This can also be sourced from the `AWS_PROFILE` environment variable.
|
|
* `shared_credentials_file` - (Optional) Path to the AWS shared credentials file. Defaults to `~/.aws/credentials`.
|
|
* `skip_credentials_validation` - (Optional) Skip credentials validation via the STS API.
|
|
* `skip_region_validation` - (Optional) Skip validation of provided region name.
|
|
* `skip_metadata_api_check` - (Optional) Skip usage of EC2 Metadata API.
|
|
* `sts_endpoint` - (Optional) Custom endpoint for the AWS Security Token Service (STS) API. This can also be sourced from the `AWS_STS_ENDPOINT` environment variable.
|
|
* `token` - (Optional) Multi-Factor Authentication (MFA) token. This can also be sourced from the `AWS_SESSION_TOKEN` environment variable.
|
|
|
|
#### Assume Role Configuration
|
|
|
|
The following configuration is optional:
|
|
|
|
* `assume_role_duration_seconds` - (Optional) Number of seconds to restrict the assume role session duration.
|
|
* `assume_role_policy` - (Optional) IAM Policy JSON describing further restricting permissions for the IAM Role being assumed.
|
|
* `assume_role_policy_arns` - (Optional) Set of Amazon Resource Names (ARNs) of IAM Policies describing further restricting permissions for the IAM Role being assumed.
|
|
* `assume_role_tags` - (Optional) Map of assume role session tags.
|
|
* `assume_role_transitive_tag_keys` - (Optional) Set of assume role session tag keys to pass to any subsequent sessions.
|
|
* `external_id` - (Optional) External identifier to use when assuming the role.
|
|
* `role_arn` - (Optional) Amazon Resource Name (ARN) of the IAM Role to assume.
|
|
* `session_name` - (Optional) Session name to use when assuming the role.
|
|
|
|
### S3 State Storage
|
|
|
|
The following configuration is required:
|
|
|
|
* `bucket` - (Required) Name of the S3 Bucket.
|
|
* `key` - (Required) Path to the state file inside the S3 Bucket. When using a non-default [workspace](/docs/language/state/workspaces.html), the state path will be `/workspace_key_prefix/workspace_name/key` (see also the `workspace_key_prefix` configuration).
|
|
|
|
The following configuration is optional:
|
|
|
|
* `acl` - (Optional) [Canned ACL](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) to be applied to the state file.
|
|
* `encrypt` - (Optional) Enable [server side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) of the state file.
|
|
* `endpoint` - (Optional) Custom endpoint for the AWS S3 API. This can also be sourced from the `AWS_S3_ENDPOINT` environment variable.
|
|
* `force_path_style` - (Optional) Enable path-style S3 URLs (`https://<HOST>/<BUCKET>` instead of `https://<BUCKET>.<HOST>`).
|
|
* `kms_key_id` - (Optional) Amazon Resource Name (ARN) of a Key Management Service (KMS) Key to use for encrypting the state.
|
|
* `sse_customer_key` - (Optional) The key to use for encrypting state with [Server-Side Encryption with Customer-Provided Keys (SSE-C)](https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html). This is the base64-encoded value of the key, which must decode to 256 bits. This can also be sourced from the `AWS_SSE_CUSTOMER_KEY` environment variable, which is recommended due to the sensitivity of the value. Setting it inside a terraform file will cause it to be persisted to disk in `terraform.tfstate`.
|
|
* `workspace_key_prefix` - (Optional) Prefix applied to the state path inside the bucket. This is only relevant when using a non-default workspace. Defaults to `env:`.
|
|
|
|
### DynamoDB State Locking
|
|
|
|
The following configuration is optional:
|
|
|
|
* `dynamodb_endpoint` - (Optional) Custom endpoint for the AWS DynamoDB API. This can also be sourced from the `AWS_DYNAMODB_ENDPOINT` environment variable.
|
|
* `dynamodb_table` - (Optional) Name of DynamoDB Table to use for state locking and consistency. The table must have a primary key named `LockID` with type of `string`. If not configured, state locking will be disabled.
|
|
|
|
## Multi-account AWS Architecture
|
|
|
|
A common architectural pattern is for an organization to use a number of
|
|
separate AWS accounts to isolate different teams and environments. For example,
|
|
a "staging" system will often be deployed into a separate AWS account than
|
|
its corresponding "production" system, to minimize the risk of the staging
|
|
environment affecting production infrastructure, whether via rate limiting,
|
|
misconfigured access controls, or other unintended interactions.
|
|
|
|
The S3 backend can be used in a number of different ways that make different
|
|
tradeoffs between convenience, security, and isolation in such an organization.
|
|
This section describes one such approach that aims to find a good compromise
|
|
between these tradeoffs, allowing use of
|
|
[Terraform's workspaces feature](/docs/language/state/workspaces.html) to switch
|
|
conveniently between multiple isolated deployments of the same configuration.
|
|
|
|
Use this section as a starting-point for your approach, but note that
|
|
you will probably need to make adjustments for the unique standards and
|
|
regulations that apply to your organization. You will also need to make some
|
|
adjustments to this approach to account for _existing_ practices within your
|
|
organization, if for example other tools have previously been used to manage
|
|
infrastructure.
|
|
|
|
Terraform is an administrative tool that manages your infrastructure, and so
|
|
ideally the infrastructure that is used by Terraform should exist outside of
|
|
the infrastructure that Terraform manages. This can be achieved by creating a
|
|
separate _administrative_ AWS account which contains the user accounts used by
|
|
human operators and any infrastructure and tools used to manage the other
|
|
accounts. Isolating shared administrative tools from your main environments
|
|
has a number of advantages, such as avoiding accidentally damaging the
|
|
administrative infrastructure while changing the target infrastructure, and
|
|
reducing the risk that an attacker might abuse production infrastructure to
|
|
gain access to the (usually more privileged) administrative infrastructure.
|
|
|
|
### Administrative Account Setup
|
|
|
|
Your administrative AWS account will contain at least the following items:
|
|
|
|
* One or more [IAM user](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_users.html)
|
|
for system administrators that will log in to maintain infrastructure in
|
|
the other accounts.
|
|
* Optionally, one or more [IAM groups](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_groups.html)
|
|
to differentiate between different groups of users that have different
|
|
levels of access to the other AWS accounts.
|
|
* An [S3 bucket](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html)
|
|
that will contain the Terraform state files for each workspace.
|
|
* A [DynamoDB table](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.TablesItemsAttributes)
|
|
that will be used for locking to prevent concurrent operations on a single
|
|
workspace.
|
|
|
|
Provide the S3 bucket name and DynamoDB table name to Terraform within the
|
|
S3 backend configuration using the `bucket` and `dynamodb_table` arguments
|
|
respectively, and configure a suitable `workspace_key_prefix` to contain
|
|
the states of the various workspaces that will subsequently be created for
|
|
this configuration.
|
|
|
|
### Environment Account Setup
|
|
|
|
For the sake of this section, the term "environment account" refers to one
|
|
of the accounts whose contents are managed by Terraform, separate from the
|
|
administrative account described above.
|
|
|
|
Your environment accounts will eventually contain your own product-specific
|
|
infrastructure. Along with this it must contain one or more
|
|
[IAM roles](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html)
|
|
that grant sufficient access for Terraform to perform the desired management
|
|
tasks.
|
|
|
|
### Delegating Access
|
|
|
|
Each Administrator will run Terraform using credentials for their IAM user
|
|
in the administrative account.
|
|
[IAM Role Delegation](http://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html)
|
|
is used to grant these users access to the roles created in each environment
|
|
account.
|
|
|
|
Full details on role delegation are covered in the AWS documentation linked
|
|
above. The most important details are:
|
|
|
|
* Each role's _Assume Role Policy_ must grant access to the administrative AWS
|
|
account, which creates a trust relationship with the administrative AWS
|
|
account so that its users may assume the role.
|
|
* The users or groups within the administrative account must also have a
|
|
policy that creates the converse relationship, allowing these users or groups
|
|
to assume that role.
|
|
|
|
Since the purpose of the administrative account is only to host tools for
|
|
managing other accounts, it is useful to give the administrative accounts
|
|
restricted access only to the specific operations needed to assume the
|
|
environment account role and access the Terraform state. By blocking all
|
|
other access, you remove the risk that user error will lead to staging or
|
|
production resources being created in the administrative account by mistake.
|
|
|
|
When configuring Terraform, use either environment variables or the standard
|
|
credentials file `~/.aws/credentials` to provide the administrator user's
|
|
IAM credentials within the administrative account to both the S3 backend _and_
|
|
to Terraform's AWS provider.
|
|
|
|
Use conditional configuration to pass a different `assume_role` value to
|
|
the AWS provider depending on the selected workspace. For example:
|
|
|
|
```hcl
|
|
variable "workspace_iam_roles" {
|
|
default = {
|
|
staging = "arn:aws:iam::STAGING-ACCOUNT-ID:role/Terraform"
|
|
production = "arn:aws:iam::PRODUCTION-ACCOUNT-ID:role/Terraform"
|
|
}
|
|
}
|
|
|
|
provider "aws" {
|
|
# No credentials explicitly set here because they come from either the
|
|
# environment or the global credentials file.
|
|
|
|
assume_role = "${var.workspace_iam_roles[terraform.workspace]}"
|
|
}
|
|
```
|
|
|
|
If workspace IAM roles are centrally managed and shared across many separate
|
|
Terraform configurations, the role ARNs could also be obtained via a data
|
|
source such as [`terraform_remote_state`](/docs/language/state/remote-state-data.html)
|
|
to avoid repeating these values.
|
|
|
|
### Creating and Selecting Workspaces
|
|
|
|
With the necessary objects created and the backend configured, run
|
|
`terraform init` to initialize the backend and establish an initial workspace
|
|
called "default". This workspace will not be used, but is created automatically
|
|
by Terraform as a convenience for users who are not using the workspaces
|
|
feature.
|
|
|
|
Create a workspace corresponding to each key given in the `workspace_iam_roles`
|
|
variable value above:
|
|
|
|
```
|
|
$ terraform workspace new staging
|
|
Created and switched to workspace "staging"!
|
|
|
|
...
|
|
|
|
$ terraform workspace new production
|
|
Created and switched to workspace "production"!
|
|
|
|
...
|
|
```
|
|
|
|
Due to the `assume_role` setting in the AWS provider configuration, any
|
|
management operations for AWS resources will be performed via the configured
|
|
role in the appropriate environment AWS account. The backend operations, such
|
|
as reading and writing the state from S3, will be performed directly as the
|
|
administrator's own user within the administrative account.
|
|
|
|
```
|
|
$ terraform workspace select staging
|
|
$ terraform apply
|
|
...
|
|
```
|
|
|
|
### Running Terraform in Amazon EC2
|
|
|
|
Teams that make extensive use of Terraform for infrastructure management
|
|
often [run Terraform in automation](https://learn.hashicorp.com/tutorials/terraform/automate-terraform?in=terraform/automation&utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS)
|
|
to ensure a consistent operating environment and to limit access to the
|
|
various secrets and other sensitive information that Terraform configurations
|
|
tend to require.
|
|
|
|
When running Terraform in an automation tool running on an Amazon EC2 instance,
|
|
consider running this instance in the administrative account and using an
|
|
[instance profile](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html)
|
|
in place of the various administrator IAM users suggested above. An IAM
|
|
instance profile can also be granted cross-account delegation access via
|
|
an IAM policy, giving this instance the access it needs to run Terraform.
|
|
|
|
To isolate access to different environment accounts, use a separate EC2
|
|
instance for each target account so that its access can be limited only to
|
|
the single account.
|
|
|
|
Similar approaches can be taken with equivalent features in other AWS compute
|
|
services, such as ECS.
|
|
|
|
### Protecting Access to Workspace State
|
|
|
|
In a simple implementation of the pattern described in the prior sections,
|
|
all users have access to read and write states for all workspaces. In many
|
|
cases it is desirable to apply more precise access constraints to the
|
|
Terraform state objects in S3, so that for example only trusted administrators
|
|
are allowed to modify the production state, or to control _reading_ of a state
|
|
that contains sensitive information.
|
|
|
|
Amazon S3 supports fine-grained access control on a per-object-path basis
|
|
using IAM policy. A full description of S3's access control mechanism is
|
|
beyond the scope of this guide, but an example IAM policy granting access
|
|
to only a single state object within an S3 bucket is shown below:
|
|
|
|
```json
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": "s3:ListBucket",
|
|
"Resource": "arn:aws:s3:::myorg-terraform-states"
|
|
},
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": ["s3:GetObject", "s3:PutObject"],
|
|
"Resource": "arn:aws:s3:::myorg-terraform-states/myapp/production/tfstate"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
It is not possible to apply such fine-grained access control to the DynamoDB
|
|
table used for locking, so it is possible for any user with Terraform access
|
|
to lock any workspace state, even if they do not have access to read or write
|
|
that state. If a malicious user has such access they could block attempts to
|
|
use Terraform against some or all of your workspaces as long as locking is
|
|
enabled in the backend configuration.
|
|
|
|
### Configuring Custom User-Agent Information
|
|
|
|
Note this feature is optional and only available in Terraform v0.13.1+.
|
|
|
|
By default, the underlying AWS client used by the Terraform AWS Provider creates requests with User-Agent headers including information about Terraform and AWS Go SDK versions. To provide additional information in the User-Agent headers, the `TF_APPEND_USER_AGENT` environment variable can be set and its value will be directly added to HTTP requests. e.g.
|
|
|
|
```sh
|
|
$ export TF_APPEND_USER_AGENT="JenkinsAgent/i-12345678 BuildID/1234 (Optional Extra Information)"
|
|
```
|