Commit Graph

305 Commits

Author SHA1 Message Date
Sander van Harmelen ea88daa499 backend/remote: add support for state locking 2018-09-10 19:49:53 +02:00
Sander van Harmelen 7c1da2ffaa
Merge pull request #18818 from hashicorp/b-associate-state
backend/remote: add the run ID to associate state
2018-09-10 19:13:53 +02:00
Sander van Harmelen cd6d75bc03 backend/remote: add the run ID to associate state
If a run ID is available, we need to make sure we pass that when creating a new state version so the state will be properly associated with the run.
2018-09-09 21:30:42 +02:00
Sander van Harmelen 3cc58813f0 backend/remote: use a search query and use pagination
To prevent making unnecessary heavy calls to the backend, we should use a search query to limit the result.

But even if we use a search query, we should still use the pagination details to make sure we retrieved all items.
2018-09-09 21:29:50 +02:00
Sander van Harmelen 62ab84a319 backend/remote: compat changes for latest `go-tfe` 2018-09-08 11:47:18 +02:00
Sander van Harmelen 3cfc83e335 backend/remote: take working directories into account
In TFE you can configure a workspace to use a custom working directory. When determining which directory that needs to be uploaded to TFE, this working directory should be taken into account to make sure we are uploading the correct root directory for the workspace.
2018-08-31 20:35:16 +02:00
Sander van Harmelen ce2869dced
Merge pull request #18760 from hashicorp/f-remote-backend
backend/migrations: migrate the default state
2018-08-30 11:16:25 +02:00
Sander van Harmelen f410a5bb26 backend/migrations: migrate the default state
Certain backends (currently only the `remote` backend) do not support using both the default and named workspaces at the same time.

To make the migration easier for users that currently use both types of workspaces, this commit adds logic to ask the user for a new workspace name during the migration process.
2018-08-29 21:37:39 +02:00
Sander van Harmelen 77b9fad7f0 backend/manta: deprecate camelcase attribute name
This change is needed (next to making it consistent as its the only attribute name that uses camelcase) because passing that attribute objectName in the update `terraform_remote_state` resource, ended up giving me this error: https://github.com/hashicorp/terraform/blob/master/helper/schema/schema.go#L736
2018-08-29 20:22:04 +02:00
Sander van Harmelen 7049d973a9 Do not use the TFE_TOKEN env variable
Instead promote the use of shared credentials using the CLI Config File
2018-08-10 20:20:16 +02:00
Sander van Harmelen 83d5f4147b backend/remote: use schema max/min items options 2018-08-05 15:30:27 +02:00
Sander van Harmelen 7fb2d1b8de Implement the Enterprise enhanced remote backend 2018-08-03 22:22:55 +02:00
Sander van Harmelen 495d1ea350 Use New() instead of `once.Do(b.init)` 2018-08-03 11:29:11 +02:00
Sander van Harmelen 97d1c46602 Update the backend import names
It’s a purely cosmetic change, but I find it easier to read them like this.
2018-08-03 11:29:11 +02:00
Kristin Laemmert 85be12d783
cli: show workspace name in destroy confirmation (#18253)
* cli: show workspace name in destroy confirmation

If the workspace name is not "default", include it in the confirmation
message for `terraform destroy`.

Fixes #15480
2018-06-19 13:35:28 -07:00
stack72 df9b2913ae backend/manta: Add support for manta backend tiered path structure
This was already added to triton-go and is now making its way to
the manta backend

```
% acctests backend/remote-state/manta
=== RUN   TestBackend_impl
--- PASS: TestBackend_impl (0.00s)
=== RUN   TestBackend
--- PASS: TestBackend (27.36s)
=== RUN   TestBackendLocked
--- PASS: TestBackendLocked (16.24s)
=== RUN   TestRemoteClient_impl
--- PASS: TestRemoteClient_impl (0.00s)
=== RUN   TestRemoteClient
--- PASS: TestRemoteClient (3.40s)
=== RUN   TestRemoteClientLocks
--- PASS: TestRemoteClientLocks (7.17s)
PASS
ok  	github.com/hashicorp/terraform/backend/remote-state/manta
```
2018-04-03 17:32:10 +03:00
James Bardin 28c46d1a90 cleanup temp files from backend tests 2018-03-28 11:00:23 -04:00
stack72 a6f76aa0af backend/manta: Manta Backend was not dealing with a ResourceNotFound
Fixes: #17314

We now deal correctly with the creation of the state file - we were
not dealing well with a ResourceNotFound error

Now that this has been changed around, we try and create the statefile
and if there is an error, we look for an existing statefile - previously
this was not the order of operations
2018-03-12 17:27:57 +02:00
Martin Atkins 6aefa5835c Merge #17218: Add -auto-approve to "terraform destroy" for consistency 2018-03-08 17:42:15 -08:00
Paul Tyng c868092d2d
Standardize http.Client creation with User-Agent 2018-02-28 12:09:50 -05:00
James Bardin e9a76808df create clistate.Locker interface
Simplify the use of clistate.Lock by creating a clistate.Locker
instance, which stores the context of locking a state, to allow unlock
to be called without knowledge of how the state was locked.

This alows the backend code to bring the needed UI methods to the point
where the state is locked, and still unlock the state from an outer
scope.

Provide a NoopLocker as well, so that callers can always call Unlock
without verifying the status of the lock.

Add the StateLocker field to the backend.Operation, so that the state
lock can be carried between the different function scopes of the backend
code. This will allow the backend context to lock the state before it's
read, while allowing the different operations to unlock the state when
they complete.
2018-02-23 16:48:15 -05:00
James Bardin 8242c773b8 missed the local state backend 2018-02-20 22:09:54 -05:00
James Bardin 57cbcbe99d update remote state tests to use new test helpers 2018-02-20 21:05:58 -05:00
James Bardin 112c37458f report the proper lock id from a state lock error
Fix the now failing state unlock test by reporting the correct ID.

The ID used by GCS is the generation number of the info object, which
isn't known until the info is already written out. While we can't get
the correct ID from the info data for the error rmessage, we can update
it with the generation number after it's read.
2018-02-20 20:54:01 -05:00
James Bardin 1fee5ae172 update gcs to use the new backend tests 2018-02-20 20:36:08 -05:00
James Bardin 85d6b1d9cc add test for lock error and force-unlock
This adds a general test to verify that a remote state backend returns
the expected error type when it cannot lock a state. It then extracts
the ID reported in the error, and attempts to unlock the state using
that ID, which simulated the force-unlock scenario. This is a separate
test, since not all backends have persistent locks that can be unlocked
later.

We also split out the backend test to be called individually as needed.
2018-02-20 20:32:07 -05:00
James Bardin ef8ed1e275 coalesce the backened interrupt code
Moves the nested select statements for backend operations into a single
function. The only difference in this part was that apply called
PersistState, which should be harmless regardless of the type of
operation being run.
2018-02-12 11:56:54 -05:00
James Bardin 7cba68326a always wait for a RunningOperation to return
If the user wishes to interrupt the running operation, only the first
interrupt was communicated to the operation by canceling the provided
context. A second interrupt would start the shutdown process, but not
communicate this to the running operation. This order of event could
cause partial writes of state.

What would happen is that once the command returns, the plugin system
would stop the provider processes. Once the provider processes dies, all
pending Eval operations would return return with an error, and quickly
cause the operation to complete. Since the backend code didn't know that
the process was shutting down imminently, it would continue by
attempting to write out the last known state. Under the right
conditions, the process would exit part way through the writing of the
state file.

Add Stop and Cancel CancelFuncs to the RunningOperation, to allow it to
easily differentiate between the two signals. The backend will then be
able to detect a shutdown and abort more gracefully.

In order to ensure that the backend is not in the process of writing the
state out, the command will always attempt to wait for the process to
complete after cancellation.
2018-02-12 11:56:03 -05:00
Laura Martin 6e1e614a56 Change -force to -auto-approve when destroying
Since an early version of Terraform, the `destroy` command has always
had the `-force` flag to allow an auto approval of the interactive
prompt. 0.11 introduced `-auto-approve` as default to `false` when using
the `apply` command.

The `-auto-approve` flag was introduced to reduce ambiguity of it's
function, but the `-force` flag was never updated for a destroy.

People often use wrappers when automating commands in Terraform, and the
inconsistency between `apply` and `destroy` means that additional logic
must be added to the wrappers to do similar functions. Both commands are
more or less able to run with similar syntax, and also heavily share
their code.

This commit updates the command in `destroy` to use the `-auto-approve` flag
making working with the Terraform CLI a more consistent experience.

We leave in `-force` in `destroy` for the time-being and flag it as
deprecated to ensure a safe switchover period.
2018-02-01 00:14:42 +00:00
James Bardin a5ed7d0ae4
Merge pull request #17086 from hashicorp/jbardin/GH-17083
only split the prefix string once from the s3 key
2018-01-17 18:51:43 -05:00
stack72 e9476c6765 backend/manta: Update manta dependencies
Internally, triton-go has changed how it handles errors. We can now get rid of
checking strings for errors, and we have introduced an errors library that
wraps some of the major errors we encounter and test for
2018-01-17 11:34:02 -08:00
Masayuki Morita f440dba137 Standardize on log level "WARN" rather than "WARNING" 2018-01-16 18:05:26 -08:00
James Bardin 43d000625d only split the prefix string once from the s3 key
Ensure that the prefix is only split off a single time when the
workspace_key_prefix is a substring of the workspace or key name.
2018-01-11 10:08:02 -05:00
Stefan Schmidt c200c170ad Handle refresh errors. 2018-01-10 16:40:20 +01:00
Paddy e4cdbd6c9f
Merge pull request #16936 from negz/gcskeys
Support 'customer supplied encryption keys' in the GCS backend
2018-01-09 01:17:35 -08:00
James Bardin 7c703f2ab2
Merge pull request #16932 from rv-jmaggio/master
Fixing issues with workspace_key_prefix for s3 backend
2018-01-05 13:24:40 -05:00
James Bardin dd5882e5b2
Merge pull request #16992 from hashicorp/jbardin/go1.10-prep
go1.10 prep
2018-01-03 15:41:41 -05:00
goraxe fb6b349e58 backend/s3: Allow forcing use of the S3 path-style API
This is useful when the underlying API is an S3-workalike service, rather than official S3.
2018-01-03 12:15:47 -08:00
Paul Stack 191cf283d5 backend/manta: Support Triton RBAC
Triton Manta allows an account other than the main triton account to be used via RBAC.

Here we expose the SDC_USER / TRITON_USER options to the backend so that a user can be specified.
2018-01-03 12:12:46 -08:00
James Bardin 4b49a323c3 go fmt
slight change to go fmt coming in 0.10
2017-12-26 13:26:38 -05:00
rv-jmaggio b313ce80c4 Changing prefix for empty workspace prefix 2017-12-19 13:14:31 -05:00
Henry Bell 56357c0ab7 Fix typo ('depracted' -> 'deprecated') 2017-12-19 16:43:06 +00:00
rv-jmaggio bef64cfe91 Fixing implementation for empty string and making acceptance test work 2017-12-19 09:31:53 -05:00
rv-jmaggio b02a1c8a46 clarifying tests and using SplitN in implementation 2017-12-18 16:24:34 -05:00
Nic Cope 011841124b Support 'customer supplied encryption keys' in the GCS backend
https://cloud.google.com/storage/docs/encryption#customer-supplied

GCS state created using customer supplied encryption keys can only be read or
modified using the same key.
2017-12-17 19:27:52 -08:00
rv-jmaggio 7f8d686074 refactor and add a test 2017-12-15 21:04:15 -05:00
rv-jmaggio 0889c118a8 Fixing issues with workspace_key_prefix 2017-12-15 17:50:36 -05:00
Rob Campbell 5daeee5f6d Update various files for new version of "stringer"
The latest version of stringer now uses strconv instead of fmt.
2017-12-11 13:26:29 -08:00
James Bardin 52eced589f accept a path or contents for credentials
Match the operation of the google provider, by accepting either a file
path or contents for both `credentials` and `GOOGLE_CREDENTIALS`
2017-12-06 18:33:59 -05:00
James Bardin 2932203492 verify that a state can be read even when locked
This should only happen when a state is loaded via the backend, as well
as a remote state.
2017-12-06 18:33:59 -05:00