Commit Graph

150 Commits

Author SHA1 Message Date
Mitchell Hashimoto 942572b574
backend/init: add atlas, remove legacy atlas remote state 2017-03-01 13:29:16 -08:00
James Bardin 7f40f90c4c remove the legacy Inmem remote state
moved to a new backend
2017-02-20 14:50:31 -05:00
James Bardin a4283d23f6 fix windows locking
Don't create a new windows handle for the lock.
2017-02-17 13:29:48 -05:00
James Bardin c0dda6a7b3 test failure for windows local locks 2017-02-17 13:29:47 -05:00
Mitchell Hashimoto 03f6c650ca
state/remote: ClientLocker is just a Client that is a state.Locker 2017-02-15 14:27:02 -08:00
Mitchell Hashimoto 9451acc5da
state/remote: add unit test to verify s3 is a ClientLocker as well 2017-02-15 14:25:53 -08:00
Mitchell Hashimoto efe754183b
state/remote: export ClientLocker, test for implementation
This adds unit tests (that will fail at compile time) if various structs
don't implement the right interfaces for locking
2017-02-15 14:20:59 -08:00
James Bardin ec00564be6 Clean up LockInfo and LockError and use them
Gove LockInfo a Marshal method for easy serialization, and a String
method for more readable output.

Have the state.Locker implementations use LockError when possible to
return LockInfo and an error.
2017-02-15 14:44:43 -05:00
James Bardin 888af93356 Have S3 check the lockID on Unlock
This needs to be improved to happen in a transaction, but it gets the
implementation passing the new tests.
2017-02-15 14:44:43 -05:00
James Bardin 67bbebce08 Have consul state reutrn the lock ID
The lock ID isn't used because the lock is tied to the client, but
return the lock ID to match the behavior of other locks.
2017-02-15 14:44:43 -05:00
James Bardin 08cff7cc13 have LocalState check Lock ID on Unlock
Have LocalState store and check the lock ID, and strictly enforce
unlocking with the correct ID.

This isn't required for local lock correctness, as we track the file
descriptor to unlock, but it does provide a varification that locking
and unlocking is done correctly throughout terraform.
2017-02-15 14:41:55 -05:00
James Bardin 6aa1066f7c Store and use the correct IDs in TestRemoteLocks 2017-02-15 14:41:55 -05:00
James Bardin f5ed8cd288 Use NewLockInfo to get a pre-populated value
Using NewLockInfo ensure we start with all required fields filled.
2017-02-15 14:41:55 -05:00
James Bardin 52b2343672 make state test pass with new state.Locker 2017-02-15 14:41:55 -05:00
James Bardin 4f0c465187 make command tests pass with new state.Locker 2017-02-15 14:41:55 -05:00
James Bardin 6d32b70637 Make S3 remote state pass tests
TODO: update S3Client to make full use of the state.Locker interface
2017-02-15 14:41:55 -05:00
James Bardin 0ad6f08204 Make remote state test run
Make them compile against the new interface.
The tests will be updated later to check new behavior.
2017-02-15 14:41:55 -05:00
James Bardin 200c8de4e9 Update the state.Locker interface
Remove CacheState rather than update it, since it's no longer used.
2017-02-15 14:41:55 -05:00
Mitchell Hashimoto a7cfb8062b
state: just style changes in the code itself (newlines, moving methods) 2017-02-14 08:51:20 -08:00
James Bardin 4d00c29706 Merge pull request #11836 from hashicorp/jbardin/state-locking
Add locking during backend configuration
2017-02-09 18:30:10 -05:00
James Bardin 0c1b138719 Add state locking during backend init
During backend initialization, especially during a migration, there is a
chance that an existing state could be overwritten.

Attempt to get a locks when writing the new state. It would be nice to
always have a lock when reading the states, but the recursive structure
of the Meta.Backend config functions makes that quite complex.
2017-02-09 15:47:27 -05:00
Daren Desjardins 5718294386 state/remote/s3: Fix Bug with Assume Role for Federated IAM Account (#10067)
* Enable remote s3 state support for assume role

- provide role_arn in backend config to enable assume role

Fixes #8739

* Check for errors after obtaining credentials
2017-02-09 16:54:49 +00:00
James Bardin 5c2e945b3c Fix format string and typos 2017-02-09 10:27:42 -05:00
James Bardin 14d965722e Use single state.LockInfo struct
Remove redundant structures
2017-02-08 11:34:31 -05:00
James Bardin 9b76f6e138 Move TestRemoteLocks to state/remote
This was legacy remote state client and backends can use this test
function without an import cycle.
2017-02-08 11:25:52 -05:00
James Bardin e92559f518 Cleanup state file during Unlock
Close and remove the file descriptor from LocalState if we Unlock the
state. Also remove an empty state file if we created it and it was never
written to. This is mostly to clean up after tests, but doesn't hurt to
not leave empty files around.
2017-02-03 18:58:18 -05:00
James Bardin a2b5811f50 Remove "expires" from lock info.
We are not going to handle lock expiration, at least at this time, so
remove the Expires fields to avoid any confusion.
2017-02-03 14:55:21 -05:00
James Bardin 1078781487 Change lock reason -> info
This makes it more apparent that the information passed in isn't
required nor will it conform to any standard. There may be call sites
that can't provide good contextual info, and we don't want to count on
that value.
2017-02-02 18:08:28 -05:00
James Bardin ebd88f8f8a add msdn link for LockFileEx 2017-02-01 12:38:10 -05:00
James Bardin 370a4ca70b add mutex for windows lockedFiles map 2017-02-01 12:36:08 -05:00
James Bardin 7590154974 Don't create empty backups
Not really a problem, but created unnecessary files and changes existing
behavior.
2017-01-30 17:16:57 -05:00
James Bardin 1646310e68 Allow a non-existent state file
A missing state file was allowed, and treated as an empty state.
2017-01-30 17:16:57 -05:00
James Bardin 3fdcbda3aa Switch from Path to PathOut on LocalState.written
After LocalState writes to a state file, we will refresh off the new
state file rather than the original Path argument.
2017-01-30 17:16:57 -05:00
James Bardin 8f7f1917f2 Remove state file data when writing a nil state
The old behavior in this situation was to simply delete the file. Since
we now have a lock on this file we don't want to close or delete it, so
instead truncate the file at offset 0.

Fix a number of related tests
2017-01-30 17:16:57 -05:00
James Bardin da0c325e5c Silence state package logs during tests
Output log output when testing is verbose
2017-01-30 17:16:57 -05:00
James Bardin 10f6d7f30f Add locking for s3 state
Use a DynamoDB table to coodinate state locking in S3.

We use a simple strategy here, defining a key containing the value of
the bucket/key of the state file as the lock. If the keys exists, the
locks fails.

TODO: decide if locks should automatically be expired, or require manual
intervention.
2017-01-30 17:16:57 -05:00
James Bardin 35307d5a60 Add remote state locking
In order to provide lockign for remote states, the Cache state,
remote.State need to expose Lock and Unlock methods. The actual locking
will be done by the remote.Client, which can implement the same
state.Locker methods.
2017-01-30 17:16:57 -05:00
James Bardin 6162cde6ff Add basic local state locking
Add the LockUnlock methods to LocalState and BackupState.

The implementation for LocalState will be platform specific. We will use
OS-native locking on the state files, speficially locking whichever
state file we intend to write to.
2017-01-30 17:16:57 -05:00
James Bardin cc0712edab add state.Locker interface
Changed from state.StateLocker to remove the stutter.

State implementations can provide Lock/Unlock methods to lock the state
file. Remote clients can also provide these same methods, which will be
called through remote.State.
2017-01-30 17:16:57 -05:00
Mitchell Hashimoto d3633ab368
state/remote: remove consul since we converted that 2017-01-26 14:33:50 -08:00
Ben Slusky e3b20c3508 Basic authentication for HTTP remote state backend 2017-01-19 21:01:50 -05:00
Aleksander Modzelewski 2627e4bd8d Add token authentication to Swift remote 2016-12-20 18:22:48 +01:00
Joe Topjian 8ecc6fda0a Merge pull request #10055 from fatmcgav/swift_add_version_expire
state/remote/swift: Add support for versioning state file in swift, and expiring objects
2016-11-22 09:57:49 -07:00
Gavin Williams c77ed69a4e state/remote/swift: Add support for versioning state file in swift, and expiring object after period of time.
Update website docs with changes
2016-11-22 16:46:10 +00:00
James Bardin 91378d0499 Merge pull request #10233 from hashicorp/jbardin/GH-10229
An empty module in state can panic
2016-11-21 17:35:33 -05:00
James Bardin 9616618de1 Make sure test has a valid ResourceState
Empty resources are now pruned more aggressively, so make sure there is
a valid ResourceState in the test ModuleState.
2016-11-20 13:13:43 -05:00
James Bardin 7715bc8423 change failing test to use subtests 2016-11-20 13:02:48 -05:00
Peter McAtominey 507efcb180 state/azure: support passing of lease ID when writing storage blob (#10115)
Also fixed tests failing auth caused by getStorageAccountAccessKey returning the
key name rather than the value

TF_ACC= go test ./state/remote -v -run=TestAz -timeout=10m -parallel=4
=== RUN   TestAzureClient_impl
--- PASS: TestAzureClient_impl (0.00s)
=== RUN   TestAzureClient
2016/11/18 13:57:34 [DEBUG] New state was assigned lineage "96037426-f95e-45c3-9183-6c39b49f590b"
2016/11/18 13:57:34 [TRACE] Preserving existing state lineage "96037426-f95e-45c3-9183-6c39b49f590b"
--- PASS: TestAzureClient (130.60s)
=== RUN   TestAzureClientEmptyLease
2016/11/18 13:59:44 [DEBUG] New state was assigned lineage "d9997445-1ebf-4b2c-b4df-15ae152f6417"
2016/11/18 13:59:44 [TRACE] Preserving existing state lineage "d9997445-1ebf-4b2c-b4df-15ae152f6417"
--- PASS: TestAzureClientEmptyLease (128.15s)
=== RUN   TestAzureClientLease
2016/11/18 14:01:55 [DEBUG] New state was assigned lineage "85912a12-2e0e-464c-9886-8add39ea3a87"
2016/11/18 14:01:55 [TRACE] Preserving existing state lineage "85912a12-2e0e-464c-9886-8add39ea3a87"
--- PASS: TestAzureClientLease (138.09s)
PASS
ok  	github.com/hashicorp/terraform/state/remote	397.111s
2016-11-18 17:26:25 +02:00
Gavin Williams bf8612b9b7 state/remote/swift: Enhancements to support full set of Openstack configuration options, plus SSL certs. Documentation updated to support 2016-11-03 07:36:18 +00:00
Gavin Williams ac07430124 state/remote/swift: Update to use Gophercloud/gophercloud, restructure to support insecure TLS, added Keystone v3 auth params
Update swift remote documentation
2016-11-01 16:55:05 +00:00