ResiLien
/
terraform
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,625 Commits 1 Branch 0 Tags 187 MiB
Commit Graph

123 Commits

Author SHA1 Message Date
Florian Forster df386d3133 backend/remote-state/gcs: Automatically create the bucket if needed. 
This resurrects the previously documented but unused "project" option.
This option is required to create buckets (so they are associated with the
right cloud project) but not to access the buckets later on (because their
names are globally unique).
 2017-10-27 16:52:21 -04:00
Florian Forster 14263223e7 backend/remote-state/gcs: Simplify initialization of the GCS client. 
This also implements the (already documented) behavior of checking the
GOOGLE_CREDENTIALS environment variable.
 2017-10-27 16:52:21 -04:00
Florian Forster 816c98f387 backend/remote-state/gcs: Read credentials with ioutil.ReadFile(). 
We never expect the raw JSON to appear in the config, so pathorcontents is
not the right package here.
 2017-10-27 16:52:21 -04:00
Florian Forster c00e929ee5 backend/remote-state/gcs: Mark the "path" option as deprecated. 2017-10-27 16:52:21 -04:00
Florian Forster 91b1a39a40 backend/remote-state/gcs: Implement an end-to-end test. 
The code is loosely based on state/remote/gcs_test.go. If the
GOOGLE_PROJECT environment variable is set, this test will

1) create a new bucket; error out if the bucket already exists.
2) create a new state
3) list states and ensure that the newly created state is listed
4) ensure that an object with the expected name exists
5) rum "state/remote".TestClient()
6) delete the state

The bucket is deleted when the test exits, though this may fail if the
bucket it not empty.
 2017-10-27 16:51:21 -04:00
Florian Forster 14bfbf0617 backend/remote-state/gcs: Document the "prefix" option. 
"state_dir" has been renamed to "prefix" to better fix the GCS
terminology.
 2017-10-27 16:51:21 -04:00
Florian Forster c054bd0939 backend/remote-state/gcs: Rename "gcloud" to "gcs" for backwards compatibility. 2017-10-27 16:51:21 -04:00
Florian Forster 93a55f15e9 backend/remote-state/gcloud: Add test for Backend.{state,lock}File(). 2017-10-27 16:51:21 -04:00
Florian Forster 5a4e2076e9 backend/remote-state/gcloud: Add the "path" config option. 
This config option was used by the legacy "gcs" client. If set, we're
using it for the default state -- all other states still use the
"state_dir" setting.
 2017-10-27 16:51:21 -04:00
Florian Forster f80b872bc3 backend/remote-state/gcloud: Unify on the "context" package. 
We don't need to use the legacy package here.
 2017-10-27 16:51:21 -04:00
Florian Forster 5d4e25ada4 backend/remote-state/gcloud: Make gcsBackend private. 
This class is only used via the "backend".Backend interface, so there is
no need to export this type beyond the gcloud package.
 2017-10-27 16:51:21 -04:00
Florian Forster 9ec39573ee backend/remote-state/gcloud: Make remoteClient private. 
This class is only used via the "state/remote".State interface, so there
is no need to export this type beyond the gcloud package.
 2017-10-27 16:51:21 -04:00
Florian Forster 42e8441a2b backend/remote-state/gcloud: Refactor Backend.State(). 
Fixes:
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
* https://github.com/golang/go/wiki/CodeReviewComments#error-strings
* https://github.com/golang/go/wiki/CodeReviewComments#initialisms
 2017-10-27 16:51:21 -04:00
Florian Forster 5cb574035a backend/remote-state/gcloud: Refactor Backend.remoteClient(). 
This replaces stateFileName() and lockFileName() with path.Join().

Fixes:
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
 2017-10-27 16:51:21 -04:00
Florian Forster 9ae45e320f backend/remote-state/gcloud: Refactor Backend.DeleteState(). 
Fixes:
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
* https://github.com/golang/go/wiki/CodeReviewComments#error-strings
 2017-10-27 16:51:21 -04:00
Florian Forster fabba5c0c8 backend/remote-state/gcloud: Refactor Backend.States(). 
The previous code listed all objects in the bucket and used local filtering
(using regular expressions) to find .tfstate objects. This new code sets
the delimiter to "/", which causes GCS to only return objects directly in
the given prefix, but not any sub"directories".

Fixes:
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
* https://github.com/golang/go/wiki/CodeReviewComments#error-strings
 2017-10-27 16:51:21 -04:00
Florian Forster 97e1aa7ce9 backend/remote-state/gcloud: Use the lock file's generation as lock ID. 
This allows Unlock() to call Delete() without reading the lock file's
content first.
 2017-10-27 16:51:21 -04:00
Florian Forster edf2096e28 backend/remote-state/gcloud: Use the context provided to configure(). 
Calling context.Background() from outside the main() function is
discouraged. The configure functions are only called from
"…/helper/schema".Backend.Configure which provides the Background context,
i.e. a long-living context we can use for backend communication.
 2017-10-27 16:51:21 -04:00
Florian Forster 2e5fca78c2 backend/remote-state/gcloud: Coding style changes. 
Refactorings to make the code more idiomatic.
 2017-10-27 16:51:21 -04:00
Florian Forster f33005faba backend/remote-state/gcloud: Handle errors returned by Write(), too. 
Many GCS errors are returned by Close(), but not all.
 2017-10-27 16:51:21 -04:00
Florian Forster 1ee194986d backend/remote-state/gcloud: Add the RemoteClient.{state,lock}File() methods. 2017-10-27 16:51:20 -04:00
Florian Forster e54d36b489 backend/remote-state/gcloud: Sort standard library imports before other imports. 2017-10-27 16:51:20 -04:00
Florian Forster 52ac764036 backend/remote-state/gcloud: Rename Url -> URL 2017-10-27 16:51:20 -04:00
Florian Forster 5313e15e07 backend/remote-state/gcloud: Move the definition of the Backend struct. 2017-10-27 16:51:20 -04:00
Florian Forster def3279ec7 backend/remote-state/gcloud: Use package provided OAuth scope. 2017-10-27 16:51:20 -04:00
Piotrek Bzdyl 5854373018 Implemented GCloud backend supporting remote locking and multiple workspaces. 2017-10-27 16:51:20 -04:00
James Bardin 31912956ce Merge pull request #16290 from hashicorp/jbardin/s3-errors 
retry on s3 state upload errors
 2017-10-17 16:04:14 -04:00
James Bardin 611b1ced59 retry on s3 state upload errors 
While #16243 added the ability to retry getting a state from S3, Put can
return the same InternalError status. Use the same retry logic when
uploading state to S3.
 2017-10-08 16:46:33 -04:00
James Bardin f5e9a20c66 reset testLockHook 2017-10-08 16:24:45 -04:00
James Bardin 25a8227291 add broken test for lock lost on connection error 
Add a way to inject network errors by setting an immediate deadline on
open consul connections. The consul client currently doesn't retry on
some errors, and will force us to lose our lock.

Once the consul api client is fixed, this test will fail.
 2017-10-08 16:16:57 -04:00
James Bardin fd9adcdb36 only init one consul client, and lower keepalive 
The consul Client is analogous to an http.Client, and we really don't
need more than 1. Configure a single client and store it in the backend.

Replace the default Transport's Dialer to reduce the KeepAlive setting
from 30s to 17s. This avoids racing with the common network timeout
value of 30s, and is also coprime to other common intervals.
 2017-10-08 11:51:35 -04:00
James Bardin d0ecb232ae record consul session ID in lock info 
This can help correlate TF and consul logs
 2017-10-08 11:24:43 -04:00
James Bardin b20ab9f0bb Merge pull request #16243 from hashicorp/jbardin/s3-errors 
retry on s3 backend internal errors
 2017-10-04 09:26:26 -04:00
bclodius fba02f0bea retry on s3 backend internal errors 
Internal errors from S3 are usually transient, and can be immediately retried.
Make 2 attempts at retreiving the state object before returning an error.
 2017-10-04 09:11:30 -04:00
James Bardin 91442b7146 Merge pull request #15680 from brunomcustodio/etcdv3-backend 
[WIP] etcd v3 backend with lock support.
 2017-10-03 14:15:58 -04:00
James Bardin d477d1f6d4 Merge pull request #15553 from bonifaido/custom_s3_backend 
Allow non-AWS S3 backends
 2017-10-02 19:39:48 -04:00
Bruno Miguel Custodio 524c3934c6
Add TLS support. 2017-09-09 08:03:59 +01:00
Bruno Miguel Custodio c8ff10f603
Reuse 'etcd' client. 2017-09-09 00:21:23 +01:00
Bruno Miguel Custodio 6daf1d9d84
Allow for username and password to be read from envvars. 2017-09-08 23:41:27 +01:00
Bruno Miguel Custodio 8f7b315037
Add documentation. 2017-09-08 23:40:05 +01:00
Bruno Miguel Custodio 54dc50ec5c
Cleanup 'etcd' after every test. 2017-09-08 22:49:23 +01:00
Bruno Miguel Custodio 3c21b9c56f
Get rid of 'keyEnvPrefix'. 2017-09-08 22:34:15 +01:00
Bruno Miguel Custodio 6570948bea
Fixing the test suite. 2017-09-08 17:46:39 +01:00
Bruno Miguel Custodio 038f5eb638
Remove 'lockSuffix' as not needed in 'etcd' v3. 2017-09-08 16:59:39 +01:00
Bruno Miguel Custodio bb4dec6032
Make sure we don't relock. 2017-09-08 16:22:14 +01:00
Bruno Miguel Custodio b8f4f6d3e2
Delete lock info when unlocking. 2017-09-08 15:21:06 +01:00
Bruno Miguel Custodio b896348230
Sort the result of 'Backend.States()'. 2017-09-08 12:25:15 +01:00
Bruno Miguel Custodio 70aad79b6e
Make 'endpoints' a schema.TypeList. 2017-09-08 12:16:00 +01:00
Bruno Miguel Custodio 52c97e9fc9
First steps towards an 'etcdv3' backend. 2017-09-08 09:43:33 +01:00
tombuildsstuff d074b0da29 Obtaining the current metadata before setting it 2017-09-06 12:41:05 -07:00