Commit Graph

193 Commits

Author SHA1 Message Date
James Bardin 91be40a577 add detailed error message s3 checksum mismatch
Have the s3 RemoteClient return a detailed error message to the user in
the case of a mismatch state checksum.
2017-05-24 13:51:33 -04:00
James Bardin 0022d224e8 store and verify s3 remote state checksum
Updates to objects in S3 are only eventually consistent. If the
RemoteClient has a DynamoDB table available, use that to store a
checksum of the last written state, so the object can be verified by the
next client to call Get.

Terraform currently doesn't have any sort of user feedback around
RefreshState/Get, so we poll only for a short time before returning an
error.
2017-05-24 13:39:30 -04:00
James Bardin b279b1abb5 check for named s3 states before acquiring a lock
In order to force-unlock a named state, we have to fetch that state
first. Don't attempt to acquire a lock if we know the state already
exists in s3.
2017-05-19 14:40:59 -04:00
James Bardin a2d452acec failing test to force-unlock a named state in s3
The State call attempts to get a lock before determining if a named
state exists. This prevents force-unlock, since we need the state to
call Unlock.
2017-05-19 14:39:42 -04:00
James Bardin 9c431aee1b only list environments when the keyName matches
Prevent extra keys in the s3 envPrefix path from showing up as
listed environments.

Better handle keys containing slashes

Add tests for unexpected keys in s3.
2017-04-12 13:57:22 -04:00
James Bardin 5621d97925 cleanup consul lock entries
This matches the consul cli behavior, where locks are cleaned up after
use.

Return an error from re-locking the state. This isn't required by the
Locker interface, but it's an added sanity check for state operations.
What was incorrect here was returning an empty ID and error, which would
indicate that Lock/Unlock isn't supported.
2017-04-06 14:19:55 -04:00
James Bardin 6e136c848a use the aws provider client initialization
Use the aws provider code to create the clients for the s3 backend, so
that all the behavior matches that of the provider.

Remove the fake creds from the test, as the aws provider will attempt to
validate them.
2017-04-05 12:39:50 -04:00
James Bardin 54aa466b74 initialize the s3 lock path in one place 2017-04-04 14:44:58 -04:00
James Bardin d059939f88 Merge pull request #13262 from hashicorp/jbardin/lock-timeouts
lock timeouts
2017-04-04 14:30:20 -04:00
James Bardin 75458a182d remove extra state.Locker assertions
All states are lockers, so get rid of extra asertions.
2017-04-01 17:01:45 -04:00
Paul Stack 4501be7e5c backend/remote-state: Add support for assume role extensions to s3 backend (#13236)
Fixes: #13234

This now matches the AWS provider for the Assume Role support
2017-03-31 19:59:29 +03:00
James Bardin 575e7f1811 Properly create a new named state in s3
If the state doesn't exist, we need to initialize one so that it can be
listed be States.
2017-03-23 10:03:22 -04:00
James Bardin fa4dc01cf4 add named state support to the s3 backend
This adds named state (environment) support to the S3 backend.

A state NAME will prepend the configured s3 key with `env:/NAME/`.
The default state will remain rooted in the bucket for backwards
compatibility.

Locks in DynamoDB use the S3 key as the as the primary key value, so
locking will work as expected for multiple states.
2017-03-22 16:59:13 -04:00
James Bardin 4980fa20e7 move s3 config from client to backend
The RemoteClient needs to be configured for the named state, so move the
general config to the backend.

Rename some fields for consistency.
2017-03-22 15:52:55 -04:00
James Bardin 9f5cf2b105 convert S3 remote state to a backend
Move the S3 State from a legacy remote state to an official backend.

This increases test coverage, uses a set schema for configuration, and
will allow new backend features to be implemented for the S3 state, e.g.
"environments".
2017-03-22 10:59:37 -04:00
Mitchell Hashimoto 0cd15176ff
backend/consul: rebase and modify gzip test 2017-03-14 17:59:54 -07:00
Mitchell Hashimoto 1daff7a826
backend/consul: support "lock" option to disable locking
This adds a "lock" config (default true) to allow users to optionally
disable state locking with Consul. This is necessary if the token given
doesn't have session permission and is necessary for backwards
compatibility.
2017-03-14 17:59:10 -07:00
James Bardin 08d2b44ada add gzip to consul backend tests 2017-03-14 15:41:00 -04:00
James Bardin 1527eae6b7 Merge pull request #8748 from christoe/christoe/8491-gzip-consul-remote-state
state/remote: Add gzip support to consul remote state (#8491)
2017-03-14 15:27:46 -04:00
James Bardin 86f711f6fc Make consul backend tests opt-in
This way we don't require contributers to have consul installed to run
make test.
2017-03-14 14:44:03 -04:00
James Bardin fa7743b627 quiet the consul server during backend tests
Don't display logs unless using `-v`
2017-03-14 10:49:38 -04:00
James Bardin 90055c6ae2 convert the consul backend to use consul/testutil
Start up our own consul server for unit tests.
2017-03-13 18:25:58 -04:00
Christoffer Kylvåg abfa35db7c backend/remote-state/consul: Make gzip compression configurable (#8491) 2017-03-13 08:17:33 +01:00
Christoffer Kylvåg e098c7c24a backend/remote-state/consul: Add gzip support to consul backend client (#8491) 2017-03-09 08:00:19 +01:00
Mitchell Hashimoto 35afb7bc86
backend/consul: some comments 2017-03-01 23:01:28 -08:00
Mitchell Hashimoto b842fd0c27
backend/consul: support named states 2017-03-01 22:58:51 -08:00
Mitchell Hashimoto 3db55cf747
backend/consul: build your own backend 2017-03-01 22:19:36 -08:00
Mitchell Hashimoto 549d525487 Merge pull request #12320 from hashicorp/b-legacy-state
command: fix loading legacy remote state + migration guide
2017-03-01 12:39:17 -08:00
Mitchell Hashimoto 6ea0dc0964
backend/consul: make address have a default to prevent input 2017-02-28 19:29:05 -08:00
James Bardin f866bb545c update remote-state/consul 2017-02-28 16:35:46 -05:00
James Bardin 597eb6c918 update remote-state/inmem client 2017-02-28 16:35:45 -05:00
James Bardin 8fdf3a42b8 update remote-state.Backend 2017-02-28 16:35:45 -05:00
James Bardin 761c63d14a Update Backend to incorporate environments
Add the missing methods/arguments to handle Terraform environments in
Backends. Extra functionality simply returns defaults for now.
2017-02-28 16:03:36 -05:00
James Bardin 2392455a67 Add inmem remote backend
Add an Inmem remote state backend.

The only config option right now is a lock_id, which will instantiate
the backend in a locked state for testing.
2017-02-20 14:49:46 -05:00
Mitchell Hashimoto efe754183b
state/remote: export ClientLocker, test for implementation
This adds unit tests (that will fail at compile time) if various structs
don't implement the right interfaces for locking
2017-02-15 14:20:59 -08:00
James Bardin ec00564be6 Clean up LockInfo and LockError and use them
Gove LockInfo a Marshal method for easy serialization, and a String
method for more readable output.

Have the state.Locker implementations use LockError when possible to
return LockInfo and an error.
2017-02-15 14:44:43 -05:00
James Bardin 67bbebce08 Have consul state reutrn the lock ID
The lock ID isn't used because the lock is tied to the client, but
return the lock ID to match the behavior of other locks.
2017-02-15 14:44:43 -05:00
James Bardin cd233fef6a make consul client pass state.Locker tests 2017-02-15 14:41:55 -05:00
James Bardin 80fab23e04 Don't test consul using demo.consul.io
We shoudn't require an external service for unit test.

TODO: create some proper acceptance tests for consul
2017-02-08 11:34:31 -05:00
James Bardin 14d965722e Use single state.LockInfo struct
Remove redundant structures
2017-02-08 11:34:31 -05:00
James Bardin 9b76f6e138 Move TestRemoteLocks to state/remote
This was legacy remote state client and backends can use this test
function without an import cycle.
2017-02-08 11:25:52 -05:00
James Bardin 54cac349a3 Add state locking to consul backend
Use consul locks to implement state locking. The lock path is state path
+ "/.lock" which matches the consul cli default for locks. Lockinfo is
stored at path + "/.lockinfo".
2017-02-08 11:25:52 -05:00
Mitchell Hashimoto 1f5d425428
backend/remote-state
This allows migration of the remote state implementations to a richer
experience including input asking.
2017-01-26 14:33:49 -08:00