Commit Graph

21339 Commits

Author SHA1 Message Date
James Bardin 6cb4e14cf8 Use pooled http client for fetching providers
While the TLS handshakes are a fairly small overhead compared to
downloading the providers, clients in some situation are failing to
complete the TLS handshake in a timely manner. It's unclear if this is
because of heavily constrained clients are stalling while doing the
major crpto operations, or the edge servers are throttling repeated
requests from the same IPs.

This should allow reusing the open TLS connection to the release edge
servers during init.
2017-10-31 10:53:42 -04:00
Radek Simko d39025b9d8
helper: Escape test name in TF_LOG_PATH_MASK 2017-10-31 06:48:57 +00:00
James Bardin a41d5d634d
Merge pull request #16498 from hashicorp/jbardin/from-module
e2e test for `init -from-module`
2017-10-30 15:56:36 -04:00
James Bardin e24acc617c force git to use https 2017-10-30 15:27:58 -04:00
Radek Simko 8ae74e66e5
Merge pull request #16500 from hashicorp/b-helper-testing-fallback
helper: Fall back to TF_LOG_PATH instead of os.Stderr
2017-10-30 19:03:56 +00:00
Loren Gordon 55d4e460de Uses the current working directory to name the built binary
This patch allows `build.sh` to be used with terraform plugins to
easily create cross-platform packages, using the same method as the
terraform Makefile:

```
mkdir scripts
curl https://raw.githubusercontent.com/hashicorp/terraform/master/scripts/build.sh -o scripts/build.sh
TF_RELEASE=1 sh -c "scripts/build.sh"  # make bin
```
2017-10-30 10:49:47 -07:00
Martin Atkins 288f7c00e4 config/module: allow registry download sources to be relative paths
It's not always easy or convenient for a web application to determine its
own absolute URL to return, so here we pragmatically allow the download
source string from a registry to be a path relative to the download
endpoint.

Since X-Terraform-Get is a go-getter string, not all valid values are
valid URLs and so we sniff for certain relative-path-looking prefixes
in order to decide whether to apply the relative lookup transform.
2017-10-30 10:37:25 -07:00
stack72 1fd0f803e4 Migrate Manta Remote state to be a backend
This PR changes manta from being a legacy remote state client to a new backend type. This also includes creating a simple lock within manta

This PR also unifies the way the triton client is configured (the schema) and also uses the same env vars to set the backend up

It is important to note that if the remote state path does not exist, then the backend will create that path. This means the user doesn't need to fall into a chicken and egg situation of creating the directory in advance before interacting with it
2017-10-30 18:36:50 +02:00
Radek Simko 48fbd60e33
helper: Fall back to TF_LOG_PATH instead of os.Stderr 2017-10-30 15:58:13 +00:00
James Bardin 53c8c1e208 e2e test for `init -from-module`
Pull down the hashicorp/vault/aws module into the current directory with
init.
2017-10-30 11:32:40 -04:00
James Bardin 4a01bf0b97
Merge pull request #16490 from hashicorp/jbardin/update-consul
update consul packages
2017-10-30 11:19:34 -04:00
James Bardin 90a7c7aed9
Merge pull request #16487 from hashicorp/jbardin/from-module
init -from-module
2017-10-30 11:13:53 -04:00
Radek Simko 35553c9ab9
Merge pull request #16491 from hashicorp/radeksimko/coc
Create CODE_OF_CONDUCT.md
2017-10-30 14:10:31 +00:00
Radek Simko 4f03de2ec4
Create CODE_OF_CONDUCT.md 2017-10-30 14:09:49 +00:00
Radek Simko 574d348f79
Merge pull request #16492 from hashicorp/radeksimko/support
Create SUPPORT.md
2017-10-30 14:05:51 +00:00
James Bardin cf54ca3b0f update tests for new consul packages
Reuse the running consul server for all tests.

Update the lostLockConnection package, since the api client should no
longer lose a lock immediately on network errors.
2017-10-29 12:40:44 -04:00
Radek Simko 38a4c9a4e0
Create SUPPORT.md 2017-10-29 08:54:48 +00:00
James Bardin 3a03d3683e update consul api packages
This is from a commit just after the v1.0.0 release, because it removes
the Porter service dependency for tests. The client api package was not
changed.
2017-10-28 19:44:10 -04:00
Greg Oliver b5d3559d2d website: refine the description of the azurerm backend 2017-10-27 17:26:37 -07:00
Jesse Adametz 643752e05a build: Stop using deprecated MAINTAINER in Dockerfile
Instead, use the "maintainer" label.
2017-10-27 17:25:44 -07:00
Martin Atkins 22c9bc811b
Update CHANGELOG.md 2017-10-27 17:13:13 -07:00
Martin Atkins 671aace8ec backend/local: disable local backup of remote state
Previously we forced all remote state backends to be wrapped in a
BackupState wrapper that generates a local "terraform.tfstate.backup"
file before updating the remote state.

This backup mechanism was motivated by allowing users to recover a
previous state if user error caused an undesirable change such as loss
of the record of one or more resources. However, it also has the downside
of flushing a possibly-sensitive state to local disk in a location where
users may not realize its purpose and accidentally check it into version
control. Those using remote state would generally prefer that state never
be flushed to local disk at all.

The use-case of recovering older states can be dealt with for remote
backends by selecting a backend that has preservation of older versions
as a first-class feature, such as S3 versioning or Terraform Enterprise's
first-class historical state versioning mechanism.

There remains still one case where state can be flushed to local disk: if
a write to the remote backend fails during "terraform apply" then we will
still create the "errored.tfstate" file to allow the user to recover. This
seems like a reasonable compromise because this is done only in an
_exceptional_ case, and the console output makes it very clear that this
file has been created.

Fixes #15339.
2017-10-27 17:06:33 -07:00
James Bardin f64851242e use Storage.GetModule for init -from-module
This will enable copying any module source into a target directory.
2017-10-27 19:27:20 -04:00
James Bardin eef2129f1a Storage.GetModule
Add GetModule for the cli to initialize from a regisry module source.

Storage.GetModule fetches a module using the same detection and
discovery as used by the normal module loading. The final copy is still
done by module.GetCopy to remove vcs files.
2017-10-27 19:17:26 -04:00
James Bardin 9df2389ba2
Merge pull request #16481 from hashicorp/jbardin/registry-auth
registry auth
2017-10-27 17:36:41 -04:00
James Bardin ac68f9a16b make testCredentials token obviously fake 2017-10-27 17:36:30 -04:00
James Bardin 76b9cac1a4 update CHANGELOG 2017-10-27 17:25:46 -04:00
James Bardin 55089e472d
Merge pull request #16484 from hashicorp/f-gcloud-backend
Convert gcloud backend
2017-10-27 17:23:22 -04:00
James Bardin c34265acac
Merge pull request #16483 from hashicorp/jbardin/consistent-s3
dynamoDB reads are not fully consisten by default
2017-10-27 17:22:50 -04:00
Florian Forster 43ce6ad95f govendor add cloud.google.com/go/storage 2017-10-27 16:52:21 -04:00
Florian Forster afa13a3d8e backend/remote-state/gcs: Move toBucketName to the tests. 2017-10-27 16:52:21 -04:00
Florian Forster 3023ae2813 backend/remote-state/gcs: Include project ID in bucket names when testing.
Since bucket names must be *globally* unique. By including the project
ID in the bucket name we ensure that people don't step on each other's
feet when testing.
2017-10-27 16:52:21 -04:00
Florian Forster dcb84ee2c2 backend/remote-state/gcs: Sanitize bucket names. 2017-10-27 16:52:21 -04:00
Florian Forster 454d6bbe2a backend/remote-state/gcs: Delete test buckets after tests complete.
This way tests clean up after themselves and don't leak buckets.
2017-10-27 16:52:21 -04:00
Florian Forster a6669c9e99 backend/remote-state/gcs: Don't enable versioning on new buckets.
Enabling versioning without setting up lifecycle management leads to
every lock file being archived, slowly accruing useless data.
2017-10-27 16:52:21 -04:00
Florian Forster a84823f597 backend/remote-state/gcs: Require TF_ACC for tests using the network. 2017-10-27 16:52:21 -04:00
Florian Forster 72ccf22a92 backend/remote-state/gcs: Implement additional tests.
This calls backend.TestBackend() and remote.TestRemoteLocks() for
standardized acceptance tests. It removes custom listing tests since
those are performed by backend.TestBackend(), too.

Since each tests uses its own bucket, all tests can be run in parallel.
2017-10-27 16:52:21 -04:00
Florian Forster 927085289d backend/remote-state/gcs: Implement the "region" config option.
This allows to select the region in which a bucket is created.
This copies behavior from the Google Cloud provider.
2017-10-27 16:52:21 -04:00
Florian Forster 9583d0945c backend/remote-state/gcs: Add support for the GOOGLE_PROJECT environment variable.
This copies behavior from the Google Cloud provider.
2017-10-27 16:52:21 -04:00
Florian Forster 37dc95158d backend/remote-state/gcs: Enable versioning on automatically created buckets. 2017-10-27 16:52:21 -04:00
Florian Forster 52e6159219 backend/remote-state/gcs: Improve "bucket" and "credentials" documentation. 2017-10-27 16:52:21 -04:00
Florian Forster df386d3133 backend/remote-state/gcs: Automatically create the bucket if needed.
This resurrects the previously documented but unused "project" option.
This option is required to create buckets (so they are associated with the
right cloud project) but not to access the buckets later on (because their
names are globally unique).
2017-10-27 16:52:21 -04:00
Florian Forster 14263223e7 backend/remote-state/gcs: Simplify initialization of the GCS client.
This also implements the (already documented) behavior of checking the
GOOGLE_CREDENTIALS environment variable.
2017-10-27 16:52:21 -04:00
Florian Forster 5205c63bc9 website/docs/backends/types/gcs.html.md: Update.
* Remove the (unused) "project" option.
* Mark the "credentials" option as optional; document behavior when
  unset.
* Mark the "path" option as deprecated (was: legacy) to match
  Terraform's terminology.
2017-10-27 16:52:21 -04:00
Florian Forster 816c98f387 backend/remote-state/gcs: Read credentials with ioutil.ReadFile().
We never expect the raw JSON to appear in the config, so pathorcontents is
not the right package here.
2017-10-27 16:52:21 -04:00
Florian Forster c00e929ee5 backend/remote-state/gcs: Mark the "path" option as deprecated. 2017-10-27 16:52:21 -04:00
Florian Forster b09f121f86 state/remote: The "gcs" client has been superseeded by the "gcs" backend. 2017-10-27 16:52:21 -04:00
Florian Forster 91b1a39a40 backend/remote-state/gcs: Implement an end-to-end test.
The code is loosely based on state/remote/gcs_test.go. If the
GOOGLE_PROJECT environment variable is set, this test will

1) create a new bucket; error out if the bucket already exists.
2) create a new state
3) list states and ensure that the newly created state is listed
4) ensure that an object with the expected name exists
5) rum "state/remote".TestClient()
6) delete the state

The bucket is deleted when the test exits, though this may fail if the
bucket it not empty.
2017-10-27 16:51:21 -04:00
Florian Forster 14bfbf0617 backend/remote-state/gcs: Document the "prefix" option.
"state_dir" has been renamed to "prefix" to better fix the GCS
terminology.
2017-10-27 16:51:21 -04:00
Florian Forster c054bd0939 backend/remote-state/gcs: Rename "gcloud" to "gcs" for backwards compatibility. 2017-10-27 16:51:21 -04:00