Commit Graph

191 Commits

Author SHA1 Message Date
James Bardin a84b4a669a use the open state file for refresh when possible
Only open a new file descriptor for RefreshState if we haven't written a
state, and don't have the correct state open already. This prevents
windows from failing to refresh a locked state.
2018-03-19 18:17:43 -04:00
James Bardin e10a7917e6 add failing test for windows state locks
Refreshing a locked state on windows could return nil if the read path
was locked, no state was yet written, and the read path is the same as
the write path.

Add a test that locks then refreshes a newly initialized state struct.
2018-03-19 17:09:53 -04:00
Paul Tyng c868092d2d
Standardize http.Client creation with User-Agent 2018-02-28 12:09:50 -05:00
jwa 3d4642719f simulate a webdav backend 2018-01-19 02:30:21 +00:00
jwa b3d432e6bc allow HTTP 201 & 204 when storing remote state
Apache mod_dav returns 201 (created) and 204 (no content) during PUTs;
treat these as valid responses when using Apache as a http backend.
2018-01-11 22:09:59 +00:00
stack72 1fd0f803e4 Migrate Manta Remote state to be a backend
This PR changes manta from being a legacy remote state client to a new backend type. This also includes creating a simple lock within manta

This PR also unifies the way the triton client is configured (the schema) and also uses the same env vars to set the backend up

It is important to note that if the remote state path does not exist, then the backend will create that path. This means the user doesn't need to fall into a chicken and egg situation of creating the directory in advance before interacting with it
2017-10-30 18:36:50 +02:00
Florian Forster b09f121f86 state/remote: The "gcs" client has been superseeded by the "gcs" backend. 2017-10-27 16:52:21 -04:00
James Bardin 36b8be43e8 use the new version package
Update all references to the version values to use the new package.
The VersionString function was left in the terraform package
specifically for the aws provider, which is vendored. We can remove that
last call once the provider is updated.
2017-10-19 21:48:08 -04:00
Martin Atkins c12d64f340 Use t.Helper() in our test helpers
Go 1.9 adds this new function which, when called, marks the caller as
being a "helper function". Helper function stack frames are then skipped
when trying to find a line of test code to blame for a test failure, so
that the code in the main test function appears in the test failure output
rather than a line within the helper function itself.

This covers many -- but probaly not all -- of our test helpers across
various packages.
2017-08-28 09:59:30 -07:00
Martin Atkins 2e0c1d07ae Merge #15793: Locking support in HTTP remote backend 2017-08-22 09:43:38 -07:00
Ross McFarland d889ac38b0 Change remote/http store to update, more consistent with doc 2017-08-20 05:59:04 -07:00
Ross McFarland 510563b67f Fully test conf handling in httpFactory 2017-08-19 11:17:25 -07:00
Ross McFarland 69546c4b33 Pass at much more flexible remote/http backend
- Configurable Put (store) method, default POST to preserve behavior
- Configurable Lock method & address
- Configurable Unlock method & address

More thorough testing still needed, but this if functional
2017-08-19 10:31:47 -07:00
James Bardin 68da0390b7 remove legacy azure remote state code
The implementation has been moved to a backend.
2017-08-17 12:57:53 -04:00
Peter McAtominey f9e8e54835 backend: convert Azure remote state to a backend
Added locking support via blob leasing (requires that an empty state is
created before any lock can be acquired.

Added support for "environments" in much the same way as the S3 backend.
2017-08-17 16:32:17 +01:00
Ross McFarland 6cdea5af5d Clean up code and make the state/http behavior more consistent 2017-08-13 09:49:08 -07:00
Ross McFarland ce4d9fb3c2 Add tests for state/http with locking 2017-08-13 09:16:42 -07:00
Ross McFarland 1d38569c91 Add Lock/Unlock support to remote/http 2017-08-13 07:39:22 -07:00
James Bardin 32ae05c342 fix strict remote.State lineage check
We can't check lineage in the remote state instance, because we may need
to overwrite a state with a new lineage. Whil it's tempting to add an
optional interface for this, like OverwriteState(), optional interfaces
are never _really_ optional, and will have to be implemented by any
wrapper types as well.

Another solution may be to add a State.Supersedes field to indicate that
we intend to replace an existing state, but that may not be worth the
extra check either.
2017-08-01 19:34:22 -04:00
James Bardin fba5decae5 update TestState helper
In practice, States must all implement the full interface, so checking
for each method set only leaves gaps where tests could be skipped.
Change the helper to only accept a full state.State implementation.

Add some Lineage, Version, and TFVersion checks to TestState to avoid
regressions.

Compare the copy test against the immediate State returnedm rather than
our previous "current" state.

Check that the states round-trip and still marhsal identically via
MarshalEqual.
2017-07-05 17:18:12 -04:00
Martin Atkins 4d53eaa6df state: more robust handling of state Serial
Previously we relied on a constellation of coincidences for everything to
work out correctly with state serials. In particular, callers needed to
be very careful about mutating states (or not) because many different bits
of code shared pointers to the same objects.

Here we move to a model where all of the state managers always use
distinct instances of state, copied when WriteState is called. This means
that they are truly a snapshot of the state as it was at that call, even
if the caller goes on mutating the state that was passed.

We also adjust the handling of serials so that the state managers ignore
any serials in incoming states and instead just treat each Persist as
the next version after what was most recently Refreshed.

(An exception exists for when nothing has been refreshed, e.g. because
we are writing a state to a location for the first time. In that case
we _do_ trust the caller, since the given state is either a new state
or it's a copy of something we're migrating from elsewhere with its
state and lineage intact.)

The intent here is to allow the rest of Terraform to not worry about
serials and state identity, and instead just treat the state as a mutable
structure. We'll just snapshot it occasionally, when WriteState is called,
and deal with serials _only_ at persist time.

This is intended as a more robust version of #15423, which was a quick
hotfix to an issue that resulted from our previous slopping handling
of state serials but arguably makes the problem worse by depending on
an additional coincidental behavior of the local backend's apply
implementation.
2017-07-05 12:34:30 -07:00
Gavin Williams 96e6bf89ec remote/swift: Remove un-used code 2017-06-27 21:18:43 +01:00
Gavin Williams ee2e390f85 remote/swift: Migrate Swift remote state to a backend
Move the Swift State from a legacy remote state to an official backend.
Add `container` and `archive_container` configuration variables, and deprecate `path` and `archive_path` variables.

Future improvements: Add support for locking and environments.
2017-06-13 22:04:01 +01:00
James Bardin 808b504bcf rename openstack provider for swift remote state 2017-06-12 13:43:51 -04:00
Tom Harvey 13583b4b8b provider/azurerm: Upgrading to AutoRest v8 (#15006)
* Upgrading to AutoRest 8

* Upgrading to AutoRest v8

* Updating the Remote State to v8
2017-06-02 15:30:41 +01:00
Tom Harvey df2875045d provider/azurerm: Upgrading to v10 of the Azure SDK (#14004)
provider/azurerm: Upgrading to v10.0.2 of the Azure SDK
2017-06-01 10:18:22 +01:00
James Bardin 4866f35645 add mutexes to Local, Backup, and InmemState 2017-05-25 11:20:52 -04:00
James Bardin f0f2220abb add mutexes to remote.State 2017-05-25 11:20:52 -04:00
Gavin Williams c63ad9c0f8 state/remote/swift: Support Openstack request logging (#13583)
* provider/openstack: Expose LogRoundTripper fields externally

* state/remote/swift: Add support for debugging Openstack calls using
OS_DEBUG env variable.

* provider/openstack: Update LogRoundTripper to log headers aswell as body.

* Add `RedactHeaders` function in order to redact sensitive http Headers.
Refactor `logRequest` and `logResponse` to use `RedactHeaders` func.
2017-04-15 17:11:28 +03:00
James Bardin eeddc3f8ea add some nil checks for unexpected lock failures 2017-04-05 09:06:47 -04:00
James Bardin af2e289212 remove Sleep from TestLockWithContext 2017-04-03 14:46:22 -04:00
James Bardin d1460d8c82 test LockWithContext 2017-04-03 11:50:19 -04:00
James Bardin 93b1dd6323 give LockWithContext a little backoff
Backoff the Lock calls exponentially, to a reasonable limit.
2017-04-01 18:01:49 -04:00
James Bardin 3f0dcd1308 Have the clistate Lock use LockWithContext
- Have the ui Lock helper use state.LockWithContext.
- Rename the message package to clistate, since that's how it's imported
  everywhere.
- Use a more idiomatic placement of the Context in the LockWithContext
  args.
2017-04-01 17:09:20 -04:00
James Bardin 826771a830 add state.LockWithContext
LockWithContext will retry a lock until the context expires or is
cancelled. This will let us implement a `-lock-timeout` flag, and make
use of existing contexts when applicable.
2017-04-01 17:01:55 -04:00
James Bardin 75458a182d remove extra state.Locker assertions
All states are lockers, so get rid of extra asertions.
2017-04-01 17:01:45 -04:00
James Bardin bf6384a163 All states are lockers
Since moving to the new backends, all states (except InmemState) are
Lockers. Add the methods to the State interface to remove a heap of
assertion checks.
2017-04-01 17:01:12 -04:00
James Bardin 9f5cf2b105 convert S3 remote state to a backend
Move the S3 State from a legacy remote state to an official backend.

This increases test coverage, uses a set schema for configuration, and
will allow new backend features to be implemented for the S3 state, e.g.
"environments".
2017-03-22 10:59:37 -04:00
Mitchell Hashimoto 1daff7a826
backend/consul: support "lock" option to disable locking
This adds a "lock" config (default true) to allow users to optionally
disable state locking with Consul. This is necessary if the token given
doesn't have session permission and is necessary for backwards
compatibility.
2017-03-14 17:59:10 -07:00
Ash Berlin 70de22253a Fix error message in Azure state backend (#12424)
The error led me to try adding `resource_group` but the code wanted `resource_group_name`. This fixes the error message to match the code.
2017-03-04 20:27:37 +02:00
Peter McAtominey cbed463b44 state/azure: add environment option for non-public cloud usage (#12364) 2017-03-02 04:58:24 +00:00
Mitchell Hashimoto 942572b574
backend/init: add atlas, remove legacy atlas remote state 2017-03-01 13:29:16 -08:00
James Bardin 7f40f90c4c remove the legacy Inmem remote state
moved to a new backend
2017-02-20 14:50:31 -05:00
James Bardin a4283d23f6 fix windows locking
Don't create a new windows handle for the lock.
2017-02-17 13:29:48 -05:00
James Bardin c0dda6a7b3 test failure for windows local locks 2017-02-17 13:29:47 -05:00
Mitchell Hashimoto 03f6c650ca
state/remote: ClientLocker is just a Client that is a state.Locker 2017-02-15 14:27:02 -08:00
Mitchell Hashimoto 9451acc5da
state/remote: add unit test to verify s3 is a ClientLocker as well 2017-02-15 14:25:53 -08:00
Mitchell Hashimoto efe754183b
state/remote: export ClientLocker, test for implementation
This adds unit tests (that will fail at compile time) if various structs
don't implement the right interfaces for locking
2017-02-15 14:20:59 -08:00
James Bardin ec00564be6 Clean up LockInfo and LockError and use them
Gove LockInfo a Marshal method for easy serialization, and a String
method for more readable output.

Have the state.Locker implementations use LockError when possible to
return LockInfo and an error.
2017-02-15 14:44:43 -05:00
James Bardin 888af93356 Have S3 check the lockID on Unlock
This needs to be improved to happen in a transaction, but it gets the
implementation passing the new tests.
2017-02-15 14:44:43 -05:00