During backend initialization, especially during a migration, there is a
chance that an existing state could be overwritten.
Attempt to get a locks when writing the new state. It would be nice to
always have a lock when reading the states, but the recursive structure
of the Meta.Backend config functions makes that quite complex.
* Enable remote s3 state support for assume role
- provide role_arn in backend config to enable assume role
Fixes#8739
* Check for errors after obtaining credentials
Close and remove the file descriptor from LocalState if we Unlock the
state. Also remove an empty state file if we created it and it was never
written to. This is mostly to clean up after tests, but doesn't hurt to
not leave empty files around.
This makes it more apparent that the information passed in isn't
required nor will it conform to any standard. There may be call sites
that can't provide good contextual info, and we don't want to count on
that value.
The old behavior in this situation was to simply delete the file. Since
we now have a lock on this file we don't want to close or delete it, so
instead truncate the file at offset 0.
Fix a number of related tests
Use a DynamoDB table to coodinate state locking in S3.
We use a simple strategy here, defining a key containing the value of
the bucket/key of the state file as the lock. If the keys exists, the
locks fails.
TODO: decide if locks should automatically be expired, or require manual
intervention.
In order to provide lockign for remote states, the Cache state,
remote.State need to expose Lock and Unlock methods. The actual locking
will be done by the remote.Client, which can implement the same
state.Locker methods.
Add the LockUnlock methods to LocalState and BackupState.
The implementation for LocalState will be platform specific. We will use
OS-native locking on the state files, speficially locking whichever
state file we intend to write to.
Changed from state.StateLocker to remove the stutter.
State implementations can provide Lock/Unlock methods to lock the state
file. Remote clients can also provide these same methods, which will be
called through remote.State.
Also fixed tests failing auth caused by getStorageAccountAccessKey returning the
key name rather than the value
TF_ACC= go test ./state/remote -v -run=TestAz -timeout=10m -parallel=4
=== RUN TestAzureClient_impl
--- PASS: TestAzureClient_impl (0.00s)
=== RUN TestAzureClient
2016/11/18 13:57:34 [DEBUG] New state was assigned lineage "96037426-f95e-45c3-9183-6c39b49f590b"
2016/11/18 13:57:34 [TRACE] Preserving existing state lineage "96037426-f95e-45c3-9183-6c39b49f590b"
--- PASS: TestAzureClient (130.60s)
=== RUN TestAzureClientEmptyLease
2016/11/18 13:59:44 [DEBUG] New state was assigned lineage "d9997445-1ebf-4b2c-b4df-15ae152f6417"
2016/11/18 13:59:44 [TRACE] Preserving existing state lineage "d9997445-1ebf-4b2c-b4df-15ae152f6417"
--- PASS: TestAzureClientEmptyLease (128.15s)
=== RUN TestAzureClientLease
2016/11/18 14:01:55 [DEBUG] New state was assigned lineage "85912a12-2e0e-464c-9886-8add39ea3a87"
2016/11/18 14:01:55 [TRACE] Preserving existing state lineage "85912a12-2e0e-464c-9886-8add39ea3a87"
--- PASS: TestAzureClientLease (138.09s)
PASS
ok github.com/hashicorp/terraform/state/remote 397.111s
- add remote state provider backed by Joyent's Manta
- add documentation of Manta remote state provider
- explicitly check for passphrase-protected SSH keys, which are currently
unsupported, and generate a more helpful error (borrowed from Packer's
solution to the same problem):
https://github.com/mitchellh/packer/blob/master/common/ssh/key.go#L27
This is a rework of pull request #6213 submitted by @joshuaspence,
adjusted to work with the remote state data source. We also add
a deprecation warning for people using the unsupported API, and retain
the ability to refer to "_local" as well as "local" for users in a mixed
version environment.
This commit enables terraform to utilise the assume role functionality
of sts to execute commands with different privileges than the API
keys specified.
Signed-off-by: Ian Duffy <ian@ianduffy.ie>
Fix checksum issue with remote state
If we read a state file with "null" objects in a module and they become
initialized to an empty map the state file may be written out with empty
objects rather than "null", changing the checksum. If we can detect
this, increment the serial number to prevent a conflict in atlas.
Our fakeAtlas test server now needs to decode the state directly rather
than using the ReadState function, so as to be able to read the state
unaltered.
The terraform.State data structures have initialization spread out
throughout the package. More thoroughly initialize State during
ReadState, and add a call to init() during WriteState as another
normalization safeguard.
Expose State.init through an exported Init() method, so that a new State
can be completely realized outside of the terraform package.
Additionally, the internal init now completely walks all internal state
structures ensuring that all maps and slices are initialized. While it
was mentioned before that the `init()` methods are problematic with too
many call sites, expanding this out better exposes the entry points that
will need to be refactored later for improved concurrency handling.
The State structures had a mix of `omitempty` fields. Remove omitempty
for all maps and slices as part of this normalization process. Make
Lineage mandatory, which is now explicitly set in some tests.
* Support setting datacenter when using consul remote state
Change-Id: I8c03f4058e9373f0de8fde7ce291ec552321cc60
* Add documentation for setting datacenter when using consul remote state
Change-Id: Ia62feea7a910a76308f0a5e7f9505c9a210e0339
* Skip IAM/STS validation and metadata check
* Skip IAM/STS identity validation - For environments or other api
implementations where there are no IAM/STS endpoints available, this
option lets you opt out from that provider initialization step.
* Skip metdata api check - For environments in which you know ahead of
time there isn't going to be a metadta api endpoint, this option lets
you opt out from that check to save time.
* Allow iam/sts initialization even if skipping account/cred validation
(#7874)
* Split out skip of IAM validation into credentials and account id
(#7874)
When refreshing remote state, indicate when no state file was found with
an ErrRemoteStateNotFound error. This prevents us from inadvertantly
getting a nil state into a terraform.State where we assume there's
always a root module.
Mitchell Hashimoto