55089e472d
Merge pull request #16484 from hashicorp/f-gcloud-backend
...
Convert gcloud backend
2017-10-27 17:23:22 -04:00
c34265acac
Merge pull request #16483 from hashicorp/jbardin/consistent-s3
...
dynamoDB reads are not fully consisten by default
2017-10-27 17:22:50 -04:00
43ce6ad95f
govendor add cloud.google.com/go/storage
2017-10-27 16:52:21 -04:00
afa13a3d8e
backend/remote-state/gcs: Move toBucketName to the tests.
2017-10-27 16:52:21 -04:00
3023ae2813
backend/remote-state/gcs: Include project ID in bucket names when testing.
...
Since bucket names must be *globally* unique. By including the project
ID in the bucket name we ensure that people don't step on each other's
feet when testing.
2017-10-27 16:52:21 -04:00
dcb84ee2c2
backend/remote-state/gcs: Sanitize bucket names.
2017-10-27 16:52:21 -04:00
454d6bbe2a
backend/remote-state/gcs: Delete test buckets after tests complete.
...
This way tests clean up after themselves and don't leak buckets.
2017-10-27 16:52:21 -04:00
a6669c9e99
backend/remote-state/gcs: Don't enable versioning on new buckets.
...
Enabling versioning without setting up lifecycle management leads to
every lock file being archived, slowly accruing useless data.
2017-10-27 16:52:21 -04:00
a84823f597
backend/remote-state/gcs: Require TF_ACC for tests using the network.
2017-10-27 16:52:21 -04:00
72ccf22a92
backend/remote-state/gcs: Implement additional tests.
...
This calls backend.TestBackend() and remote.TestRemoteLocks() for
standardized acceptance tests. It removes custom listing tests since
those are performed by backend.TestBackend(), too.
Since each tests uses its own bucket, all tests can be run in parallel.
2017-10-27 16:52:21 -04:00
927085289d
backend/remote-state/gcs: Implement the "region" config option.
...
This allows to select the region in which a bucket is created.
This copies behavior from the Google Cloud provider.
2017-10-27 16:52:21 -04:00
9583d0945c
backend/remote-state/gcs: Add support for the GOOGLE_PROJECT environment variable.
...
This copies behavior from the Google Cloud provider.
2017-10-27 16:52:21 -04:00
37dc95158d
backend/remote-state/gcs: Enable versioning on automatically created buckets.
2017-10-27 16:52:21 -04:00
52e6159219
backend/remote-state/gcs: Improve "bucket" and "credentials" documentation.
2017-10-27 16:52:21 -04:00
df386d3133
backend/remote-state/gcs: Automatically create the bucket if needed.
...
This resurrects the previously documented but unused "project" option.
This option is required to create buckets (so they are associated with the
right cloud project) but not to access the buckets later on (because their
names are globally unique).
2017-10-27 16:52:21 -04:00
14263223e7
backend/remote-state/gcs: Simplify initialization of the GCS client.
...
This also implements the (already documented) behavior of checking the
GOOGLE_CREDENTIALS environment variable.
2017-10-27 16:52:21 -04:00
5205c63bc9
website/docs/backends/types/gcs.html.md: Update.
...
* Remove the (unused) "project" option.
* Mark the "credentials" option as optional; document behavior when
unset.
* Mark the "path" option as deprecated (was: legacy) to match
Terraform's terminology.
2017-10-27 16:52:21 -04:00
816c98f387
backend/remote-state/gcs: Read credentials with ioutil.ReadFile().
...
We never expect the raw JSON to appear in the config, so pathorcontents is
not the right package here.
2017-10-27 16:52:21 -04:00
c00e929ee5
backend/remote-state/gcs: Mark the "path" option as deprecated.
2017-10-27 16:52:21 -04:00
b09f121f86
state/remote: The "gcs" client has been superseeded by the "gcs" backend.
2017-10-27 16:52:21 -04:00
91b1a39a40
backend/remote-state/gcs: Implement an end-to-end test.
...
The code is loosely based on state/remote/gcs_test.go. If the
GOOGLE_PROJECT environment variable is set, this test will
1) create a new bucket; error out if the bucket already exists.
2) create a new state
3) list states and ensure that the newly created state is listed
4) ensure that an object with the expected name exists
5) rum "state/remote".TestClient()
6) delete the state
The bucket is deleted when the test exits, though this may fail if the
bucket it not empty.
2017-10-27 16:51:21 -04:00
14bfbf0617
backend/remote-state/gcs: Document the "prefix" option.
...
"state_dir" has been renamed to "prefix" to better fix the GCS
terminology.
2017-10-27 16:51:21 -04:00
c054bd0939
backend/remote-state/gcs: Rename "gcloud" to "gcs" for backwards compatibility.
2017-10-27 16:51:21 -04:00
93a55f15e9
backend/remote-state/gcloud: Add test for Backend.{state,lock}File().
2017-10-27 16:51:21 -04:00
5a4e2076e9
backend/remote-state/gcloud: Add the "path" config option.
...
This config option was used by the legacy "gcs" client. If set, we're
using it for the default state -- all other states still use the
"state_dir" setting.
2017-10-27 16:51:21 -04:00
f80b872bc3
backend/remote-state/gcloud: Unify on the "context" package.
...
We don't need to use the legacy package here.
2017-10-27 16:51:21 -04:00
5d4e25ada4
backend/remote-state/gcloud: Make gcsBackend private.
...
This class is only used via the "backend".Backend interface, so there is
no need to export this type beyond the gcloud package.
2017-10-27 16:51:21 -04:00
9ec39573ee
backend/remote-state/gcloud: Make remoteClient private.
...
This class is only used via the "state/remote".State interface, so there
is no need to export this type beyond the gcloud package.
2017-10-27 16:51:21 -04:00
42e8441a2b
backend/remote-state/gcloud: Refactor Backend.State().
...
Fixes:
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
* https://github.com/golang/go/wiki/CodeReviewComments#error-strings
* https://github.com/golang/go/wiki/CodeReviewComments#initialisms
2017-10-27 16:51:21 -04:00
5cb574035a
backend/remote-state/gcloud: Refactor Backend.remoteClient().
...
This replaces stateFileName() and lockFileName() with path.Join().
Fixes:
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
2017-10-27 16:51:21 -04:00
9ae45e320f
backend/remote-state/gcloud: Refactor Backend.DeleteState().
...
Fixes:
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
* https://github.com/golang/go/wiki/CodeReviewComments#error-strings
2017-10-27 16:51:21 -04:00
fabba5c0c8
backend/remote-state/gcloud: Refactor Backend.States().
...
The previous code listed all objects in the bucket and used local filtering
(using regular expressions) to find .tfstate objects. This new code sets
the delimiter to "/", which causes GCS to only return objects directly in
the given prefix, but not any sub"directories".
Fixes:
* https://github.com/golang/go/wiki/CodeReviewComments#doc-comments
* https://github.com/golang/go/wiki/CodeReviewComments#error-strings
2017-10-27 16:51:21 -04:00
97e1aa7ce9
backend/remote-state/gcloud: Use the lock file's generation as lock ID.
...
This allows Unlock() to call Delete() without reading the lock file's
content first.
2017-10-27 16:51:21 -04:00
edf2096e28
backend/remote-state/gcloud: Use the context provided to configure().
...
Calling context.Background() from outside the main() function is
discouraged. The configure functions are only called from
"…/helper/schema".Backend.Configure which provides the Background context,
i.e. a long-living context we can use for backend communication.
2017-10-27 16:51:21 -04:00
2e5fca78c2
backend/remote-state/gcloud: Coding style changes.
...
Refactorings to make the code more idiomatic.
2017-10-27 16:51:21 -04:00
f33005faba
backend/remote-state/gcloud: Handle errors returned by Write(), too.
...
Many GCS errors are returned by Close(), but not all.
2017-10-27 16:51:21 -04:00
1ee194986d
backend/remote-state/gcloud: Add the RemoteClient.{state,lock}File() methods.
2017-10-27 16:51:20 -04:00
e54d36b489
backend/remote-state/gcloud: Sort standard library imports before other imports.
2017-10-27 16:51:20 -04:00
52ac764036
backend/remote-state/gcloud: Rename Url -> URL
2017-10-27 16:51:20 -04:00
5313e15e07
backend/remote-state/gcloud: Move the definition of the Backend struct.
2017-10-27 16:51:20 -04:00
def3279ec7
backend/remote-state/gcloud: Use package provided OAuth scope.
2017-10-27 16:51:20 -04:00
5854373018
Implemented GCloud backend supporting remote locking and multiple workspaces.
2017-10-27 16:51:20 -04:00
b040cd0837
dynamoDB reads are not fully consisten by default
...
Use fully consistent reads for backend operations.
2017-10-27 16:43:30 -04:00
ee56e3226b
test that credentials are added to registry reqs
...
Add the missing set in lookupModuleLocation
2017-10-27 16:16:35 -04:00
5203c66116
pass command credentials into module.Storage
2017-10-27 16:16:35 -04:00
4e8fe97556
add credentials to module.Storage
...
Provide a way to pass in credentials to be used by the module.Storage
when contacting registries.
Remove the mockTLSServer and use a static discovery map pointing to the
http url for tests.
2017-10-27 16:16:35 -04:00
633d428c15
Merge pull request #16480 from hashicorp/0.11-dev
...
0.11 dev
2017-10-27 16:02:19 -04:00
3a495ffe56
rename ModuleStorage to Storage
...
get rid of stutter and use module.Storage
2017-10-27 13:11:21 -04:00
70a5b1b734
make terraform work with ModuleStorage
2017-10-27 13:06:07 -04:00
f2a7b94692
use the new ModuleStorage in the command package
...
Update the command package to use the new module storage. Move the old
command output strings into the module storage itself. This could be
moved back later either by using ui callbacks, or designing a module
storage interface once we know what the final requirements will look
like.
2017-10-27 12:58:24 -04:00
James Bardin