Commit Graph

269 Commits

Author SHA1 Message Date
James Bardin 7c703f2ab2
Merge pull request #16932 from rv-jmaggio/master
Fixing issues with workspace_key_prefix for s3 backend
2018-01-05 13:24:40 -05:00
James Bardin dd5882e5b2
Merge pull request #16992 from hashicorp/jbardin/go1.10-prep
go1.10 prep
2018-01-03 15:41:41 -05:00
goraxe fb6b349e58 backend/s3: Allow forcing use of the S3 path-style API
This is useful when the underlying API is an S3-workalike service, rather than official S3.
2018-01-03 12:15:47 -08:00
Paul Stack 191cf283d5 backend/manta: Support Triton RBAC
Triton Manta allows an account other than the main triton account to be used via RBAC.

Here we expose the SDC_USER / TRITON_USER options to the backend so that a user can be specified.
2018-01-03 12:12:46 -08:00
James Bardin 4b49a323c3 go fmt
slight change to go fmt coming in 0.10
2017-12-26 13:26:38 -05:00
rv-jmaggio b313ce80c4 Changing prefix for empty workspace prefix 2017-12-19 13:14:31 -05:00
Henry Bell 56357c0ab7 Fix typo ('depracted' -> 'deprecated') 2017-12-19 16:43:06 +00:00
rv-jmaggio bef64cfe91 Fixing implementation for empty string and making acceptance test work 2017-12-19 09:31:53 -05:00
rv-jmaggio b02a1c8a46 clarifying tests and using SplitN in implementation 2017-12-18 16:24:34 -05:00
rv-jmaggio 7f8d686074 refactor and add a test 2017-12-15 21:04:15 -05:00
rv-jmaggio 0889c118a8 Fixing issues with workspace_key_prefix 2017-12-15 17:50:36 -05:00
Rob Campbell 5daeee5f6d Update various files for new version of "stringer"
The latest version of stringer now uses strconv instead of fmt.
2017-12-11 13:26:29 -08:00
James Bardin 52eced589f accept a path or contents for credentials
Match the operation of the google provider, by accepting either a file
path or contents for both `credentials` and `GOOGLE_CREDENTIALS`
2017-12-06 18:33:59 -05:00
James Bardin 2932203492 verify that a state can be read even when locked
This should only happen when a state is loaded via the backend, as well
as a remote state.
2017-12-06 18:33:59 -05:00
James Bardin 9dea2f78d4 create unique buckets for each test, and clean up
This creates a unique bucket name for each test, so that the tests in
parallel don't collide, and buckets left over from interrupted tests
don't cause future failures.

Also make sure that buckets are removed, regardless of content.
2017-12-06 18:33:59 -05:00
James Bardin aec45e6967 backends should never create unmanaged resources
The backend was creating bucket named in the configuration if it didn't
exist. We don't allow other backends to do this, because these are not
managed resources that terraform can control.
2017-12-06 18:32:41 -05:00
Kaveh Mousavi Zamani 7507e3cd21 backend/gcs: fix locking issue when used with terraform_remote_state
Previously there was a problem with double-locking when using the GCS backend with the terraform_remote_state data source.

Here we adjust the locking methodology to avoid that problem.
2017-12-06 09:36:16 -08:00
James Bardin 12b7dac124
Merge pull request #16833 from hashicorp/jbardin/plan-shutdown
Fully enable shutdown for plan and refresh in the local backend
2017-12-05 16:48:34 -05:00
Sander van Harmelen aaedf255c3 backend/s3: allow skipping the region check (#16757)
Without the possibility to skip this check, it’s not possible to use a custom region with a third-party service that mimicks the S3 API.
2017-12-05 11:19:36 -08:00
James Bardin 85295e5c23 watch for cancellation in plan and refresh
Cancellation in the local backend was only implemented for apply.
2017-12-05 10:17:20 -05:00
Martin Atkins ba0514106a return tfdiags.Diagnostics from validation methods
Validation is the best time to return detailed diagnostics
to the user since we're much more likely to have source
location information, etc than we are in later operations.

This change doesn't actually add any detail to the messages
yet, but it changes the interface so that we can gradually
introduce more detailed diagnostics over time.

While here there are some minor adjustments to some of the
messages to improve their consistency with terminology we
use elsewhere.
2017-11-28 11:15:29 -08:00
James Bardin 00b7715710
Merge pull request #16586 from hashicorp/jbardin/providers
Store resolved providers in state
2017-11-08 14:27:48 -05:00
Nic Cope 27ba7de77d backend/gcs: automatically add trailing slash to prefix
The backend doesn't function correctly if a trailing slash is not present, due to how workspaces are enumerated.
2017-11-08 11:24:36 -08:00
James Bardin 09180a10ff cannot destroy without a config 2017-11-07 21:23:37 -05:00
James Bardin d62e9217ae update test state strings for backend/local 2017-11-07 21:23:10 -05:00
Malik Bougacha ed0d57d1b7 Get the enviroment from the environment variable (#16326)
* Get the enviroment from the enviroment variable

* typo
2017-11-06 16:59:48 +00:00
stack72 1fd0f803e4 Migrate Manta Remote state to be a backend
This PR changes manta from being a legacy remote state client to a new backend type. This also includes creating a simple lock within manta

This PR also unifies the way the triton client is configured (the schema) and also uses the same env vars to set the backend up

It is important to note that if the remote state path does not exist, then the backend will create that path. This means the user doesn't need to fall into a chicken and egg situation of creating the directory in advance before interacting with it
2017-10-30 18:36:50 +02:00
James Bardin cf54ca3b0f update tests for new consul packages
Reuse the running consul server for all tests.

Update the lostLockConnection package, since the api client should no
longer lose a lock immediately on network errors.
2017-10-29 12:40:44 -04:00
James Bardin 3a03d3683e update consul api packages
This is from a commit just after the v1.0.0 release, because it removes
the Porter service dependency for tests. The client api package was not
changed.
2017-10-28 19:44:10 -04:00
Martin Atkins 671aace8ec backend/local: disable local backup of remote state
Previously we forced all remote state backends to be wrapped in a
BackupState wrapper that generates a local "terraform.tfstate.backup"
file before updating the remote state.

This backup mechanism was motivated by allowing users to recover a
previous state if user error caused an undesirable change such as loss
of the record of one or more resources. However, it also has the downside
of flushing a possibly-sensitive state to local disk in a location where
users may not realize its purpose and accidentally check it into version
control. Those using remote state would generally prefer that state never
be flushed to local disk at all.

The use-case of recovering older states can be dealt with for remote
backends by selecting a backend that has preservation of older versions
as a first-class feature, such as S3 versioning or Terraform Enterprise's
first-class historical state versioning mechanism.

There remains still one case where state can be flushed to local disk: if
a write to the remote backend fails during "terraform apply" then we will
still create the "errored.tfstate" file to allow the user to recover. This
seems like a reasonable compromise because this is done only in an
_exceptional_ case, and the console output makes it very clear that this
file has been created.

Fixes #15339.
2017-10-27 17:06:33 -07:00
James Bardin 55089e472d
Merge pull request #16484 from hashicorp/f-gcloud-backend
Convert gcloud backend
2017-10-27 17:23:22 -04:00
Florian Forster afa13a3d8e backend/remote-state/gcs: Move toBucketName to the tests. 2017-10-27 16:52:21 -04:00
Florian Forster 3023ae2813 backend/remote-state/gcs: Include project ID in bucket names when testing.
Since bucket names must be *globally* unique. By including the project
ID in the bucket name we ensure that people don't step on each other's
feet when testing.
2017-10-27 16:52:21 -04:00
Florian Forster dcb84ee2c2 backend/remote-state/gcs: Sanitize bucket names. 2017-10-27 16:52:21 -04:00
Florian Forster 454d6bbe2a backend/remote-state/gcs: Delete test buckets after tests complete.
This way tests clean up after themselves and don't leak buckets.
2017-10-27 16:52:21 -04:00
Florian Forster a6669c9e99 backend/remote-state/gcs: Don't enable versioning on new buckets.
Enabling versioning without setting up lifecycle management leads to
every lock file being archived, slowly accruing useless data.
2017-10-27 16:52:21 -04:00
Florian Forster a84823f597 backend/remote-state/gcs: Require TF_ACC for tests using the network. 2017-10-27 16:52:21 -04:00
Florian Forster 72ccf22a92 backend/remote-state/gcs: Implement additional tests.
This calls backend.TestBackend() and remote.TestRemoteLocks() for
standardized acceptance tests. It removes custom listing tests since
those are performed by backend.TestBackend(), too.

Since each tests uses its own bucket, all tests can be run in parallel.
2017-10-27 16:52:21 -04:00
Florian Forster 927085289d backend/remote-state/gcs: Implement the "region" config option.
This allows to select the region in which a bucket is created.
This copies behavior from the Google Cloud provider.
2017-10-27 16:52:21 -04:00
Florian Forster 9583d0945c backend/remote-state/gcs: Add support for the GOOGLE_PROJECT environment variable.
This copies behavior from the Google Cloud provider.
2017-10-27 16:52:21 -04:00
Florian Forster 37dc95158d backend/remote-state/gcs: Enable versioning on automatically created buckets. 2017-10-27 16:52:21 -04:00
Florian Forster df386d3133 backend/remote-state/gcs: Automatically create the bucket if needed.
This resurrects the previously documented but unused "project" option.
This option is required to create buckets (so they are associated with the
right cloud project) but not to access the buckets later on (because their
names are globally unique).
2017-10-27 16:52:21 -04:00
Florian Forster 14263223e7 backend/remote-state/gcs: Simplify initialization of the GCS client.
This also implements the (already documented) behavior of checking the
GOOGLE_CREDENTIALS environment variable.
2017-10-27 16:52:21 -04:00
Florian Forster 816c98f387 backend/remote-state/gcs: Read credentials with ioutil.ReadFile().
We never expect the raw JSON to appear in the config, so pathorcontents is
not the right package here.
2017-10-27 16:52:21 -04:00
Florian Forster c00e929ee5 backend/remote-state/gcs: Mark the "path" option as deprecated. 2017-10-27 16:52:21 -04:00
Florian Forster 91b1a39a40 backend/remote-state/gcs: Implement an end-to-end test.
The code is loosely based on state/remote/gcs_test.go. If the
GOOGLE_PROJECT environment variable is set, this test will

1) create a new bucket; error out if the bucket already exists.
2) create a new state
3) list states and ensure that the newly created state is listed
4) ensure that an object with the expected name exists
5) rum "state/remote".TestClient()
6) delete the state

The bucket is deleted when the test exits, though this may fail if the
bucket it not empty.
2017-10-27 16:51:21 -04:00
Florian Forster 14bfbf0617 backend/remote-state/gcs: Document the "prefix" option.
"state_dir" has been renamed to "prefix" to better fix the GCS
terminology.
2017-10-27 16:51:21 -04:00
Florian Forster c054bd0939 backend/remote-state/gcs: Rename "gcloud" to "gcs" for backwards compatibility. 2017-10-27 16:51:21 -04:00
Florian Forster 93a55f15e9 backend/remote-state/gcloud: Add test for Backend.{state,lock}File(). 2017-10-27 16:51:21 -04:00
Florian Forster 5a4e2076e9 backend/remote-state/gcloud: Add the "path" config option.
This config option was used by the legacy "gcs" client. If set, we're
using it for the default state -- all other states still use the
"state_dir" setting.
2017-10-27 16:51:21 -04:00