Commit Graph

20882 Commits

Author SHA1 Message Date
Christoph Kappel af206e7543 Clean clients from chef-vault on recreate_client enabled (#16357)
Fixes #15921

When terraform re-creates an existing node/client with chef provisioner,
the already existing client (which has old keys) must be removed from
the vault items. Afterwards, the chef-vault will be updated with the
newly created client (which has the new keys). Therefore, the recreated
client will be able to decrypt the vault items properly.
2017-10-20 17:46:29 +02:00
James Bardin 36b8be43e8 use the new version package
Update all references to the version values to use the new package.
The VersionString function was left in the terraform package
specifically for the aws provider, which is vendored. We can remove that
last call once the provider is updated.
2017-10-19 21:48:08 -04:00
James Bardin 3f6a7f3ed2 add a terraform version package
Many terraform project packages now want access to the current terraform
version information. Since most packages end up being consumed by the
core terraform package, preventing import cycles becomes difficult.
2017-10-19 19:18:17 -04:00
Martin Atkins d5b0beac01 svchost/auth: static credentials source
This uses an in-memory table of credentials keyed on hostname. This is
the simplest possible credentials source that can actually return
credentials, and is suitable for representing statically-configured
credentials from configuration.
2017-10-19 11:18:43 -07:00
Martin Atkins 865e61b4ea main+command: provide service disco and creds to commands
The command package is the main place we need access to these, so that
we can use them during init (to install packages, for example) and so that
we can use them to configure remote backends.

For the moment we're just providing an empty credentials object, which
will start to include both statically-configured and
helper-program-provided credentials sources in subsequent commits.
2017-10-19 11:18:43 -07:00
Martin Atkins 0db521a281 svchost/auth: expose a "NoCredentials" credentials source
For situations where no credentials are needed but where a working
CredentialsSource is still required, this variable provides a convenient
way to get a fully-functional-but-empty credentials source.
2017-10-19 11:18:43 -07:00
Martin Atkins 83b098344b svchost/disco: add credentials, if available, to disco requests
Although service discovery metadata is usually not sensitive, a service
host may wish to produce different results depending on the requesting
user, such as if users are migrating between two different implementations
that are both running concurrently for some period.
2017-10-19 11:18:43 -07:00
Martin Atkins fcff4cbc95 svchost/auth: caching credentials source
This credentials source wraps another and caches its results in memory
in a map that is keyed on the request hostname.

This should ease the common case of many services operating on the same
hostname by allowing an initial set of credentials to be re-used for
subsequent requests.
2017-10-19 11:18:43 -07:00
Martin Atkins 981c95f699 svchost/auth: CredentialsSource that runs an external program
This CredentialsSource can serve as an extension point to pass credentials
from an arbitrary external system to Terraform. For example, an external
helper program could fetch limited-time credentials from HashiCorp Vault
and return them, thus avoiding the need for any static configuration to
be maintained locally (except a Vault token!).

So far there are no real programs implementing this protocol, though this
commit includes a basic implementation that we use for unit tests.
2017-10-19 11:18:43 -07:00
Martin Atkins 1b60e8fdb6 svchost/auth: HostCredentialsFromMap function
This function deals with turning a map derived from some user input
(e.g. in a config file) into a HostCredentials object, if possible. This
will be used as a standard way to specify credentials so we have a place
to add new credentials types in future and have support for those across
all of our map-based CredentialsSources.
2017-10-19 11:18:43 -07:00
Martin Atkins 43a3357473 svchost/auth: token-based HostCredentials
This is the only credentials type we support right now, which just sends
an opaque token via the "Bearer" HTTP auth scheme.
2017-10-19 11:18:43 -07:00
Martin Atkins 3c65b5dd61 svchost/auth: package for looking up credentials for service hosts
This commit establishes the primary types involved here. Subsequent
commits will add specific implementations of these interfaces.
2017-10-19 11:18:43 -07:00
Martin Atkins 6cd9a8f9c2 svchost/disco: lookup of service URLs within a discovered map
This package implements our Terraform-native Service discovery protocol,
which allows us to find the base URL for a particular service given a
hostname that was already validated and normalized by the svchost package.
2017-10-19 11:18:43 -07:00
Martin Atkins db08ee4ac5 svchost: new package for wrangling service hostnames
We're starting to expose a number of so-called "Terraform-native services"
that can be offered under a friendly hostname. The first of these will
be module registry services, as they expand from the public
Terraform Registry to private registry services within Terraform
Enterprise and elsewhere.

This package is for wrangling these "friendly hostnames", which start
their lives as user-specified unicode strings, can be converted to
Punycode for storage and comparison, and can in turn be converted back
into normalized unicode for display to the user.
2017-10-19 11:18:43 -07:00
Martin Atkins 9bd54c24e7 govendor fetch golang.org/x/net/idna 2017-10-19 11:18:43 -07:00
Nick Fagerlund 7d9152850a website: Add nav items for Terraform Recommended Practices guide to guides.erb (#16398) 2017-10-19 11:43:15 -05:00
James Bardin 31912956ce Merge pull request #16290 from hashicorp/jbardin/s3-errors
retry on s3 state upload errors
2017-10-17 16:04:14 -04:00
James Bardin 344fa54638 Merge pull request #16289 from hashicorp/jbardin/consul-lock
some consul lock improvements
2017-10-17 15:32:26 -04:00
devonbleak 3253367c64 Fix alphabetical ordering in providers sidebar 2017-10-17 07:40:45 -07:00
Matt McQuillan 02404238a4 Updating go-checkpoint lib to have a fixed timeout 2017-10-17 07:25:56 -07:00
Aleksejs Sinicins ece099f453 website: fix "contains" interpolation example 2017-10-17 07:25:09 -07:00
Martin Atkins 183833affc core: terraform.ResourceProvider.GetSchema method
In order to parse provider, resource and data source configuration from
HCL2 config files, we need to know the relevant configuration schema.
This new method allows Terraform Core to request these from a provider.

This is a breaking change to this interface, so all of its implementers
in this package are updated too. This includes concrete implementations
of the new method in helper/schema that use the schema conversion code
added in an earlier commit to produce a configschema.Block automatically.

Plugins compiled against prior versions of helper/schema will not have
support for this method, and so calls to them will fail. Callers of
this new method will therefore need to sniff for support using the
SchemaAvailable field added to both ResourceType and DataSource.

This careful handling will need to persist until next time we increment
the plugin protocol version, at which point we can make the breaking
change of requiring this information to be available.
2017-10-17 07:23:41 -07:00
Martin Atkins ccb328cc1f config: source code ranges for InterpolatedVariable
Having a reference to the originating source range will allow us to
generate decent error messages if certain references can't be resolved
at interpolation time.

This is not yet populated or used. It will never be populated nor used by
the current HCL/HIL-based interpolation path, but will be used by the
experimental HCL2-based interpolation path to give it the necessary info
to produce diagnostics.
2017-10-17 07:20:17 -07:00
Martin Atkins 71e989ba3e config/hcl2shim: make some of the HCL2 shim functions public
The value-conversion machinery is also needed in the main "terraform"
package to help us populate our HCL2 evaluation scope, so a subset of the
shim functions move here into a new package where they can be public.

Some of them remain private within the config package since they depend
on some other symbols in the config package, and they are not needed
by outside callers anyway.
2017-10-16 17:54:02 -07:00
Martin Atkins 094cdca688 tfdiags: show descriptions in diagnosticsAsError
Previously we were showing only the summaries when converting to a string
error, but HCL generates summaries that indicate only the _type_ of error,
expecting that the detail will give the details, and so we need to show
both in order to produce a useful error message.
2017-10-16 17:53:06 -07:00
Martin Atkins 6f7bc4f5d7 command: use c.showDiagnostics for backend operation errors
This allows richer diagnostics produced by some subsystems to be displayed
in full-fidelity to the user.
2017-10-16 17:53:06 -07:00
Martin Atkins 22fb82963c config: when copying a HCL2 RawConfig, don't corrupt it
Previously we were demoting HCL2 RawConfigs into empty old-school
RawConfigs on copy.
2017-10-16 17:52:23 -07:00
Martin Atkins b5403059e6 scripts: helper for launching Terraform in the dlv debugger
This launches Terraform inside a headless dlv configured to accept a
remote debugging process. It's configured this way so it can be easily
used from a debugger GUI integrated into an IDE/editor, but it can also
be used from the CLI by running the command it prints.

Using a remote debugger here is useful even when debugging with the CLI,
since it keeps Terraform's verbose and colorful output from interfering
with the debugger UI.
2017-10-16 17:51:54 -07:00
Martin Atkins 26861dd7aa govendor fetch github.com/zclconf/go-cty/...
This new version supports a conversion from object types to map types,
which is important for Terraform because HCL2 { ... } syntax produces
objects but lots of Terraform attributes require maps.
2017-10-16 17:51:37 -07:00
Martin Atkins c007e3a7da tfdiags: Helper to construct SourceRange from hcl.Range
HCL will be the most frequent origin of a source range, so this helper
should make it easier to translate these between the two worlds so we
can more easily use them in native Terraform diagnostics and other
messaging.
2017-10-16 17:51:16 -07:00
Martin Atkins ccc20fdad1 helper/didyoumean: helper library for "Did you mean ...? suggestions
Uses Levenshtein distance to decide if the input is similar enough to one
of the given suggestions, and returns that suggestion if so.

The distance threshold of three was arrived at experimentally, and has
no objective basis.
2017-10-16 17:50:57 -07:00
Radek Simko 0388bc1b23 Merge pull request #16356 from hashicorp/t-log-path-mask
helper: Allow logs isolation per acceptance test
2017-10-16 20:32:38 +03:00
Radek Simko 4052a8f956
helper: Allow logs isolation per acceptance test 2017-10-14 16:39:11 +03:00
Chris Griggs 5fb1fe9de8 Update README guidelines for module registry (#16223) 2017-10-12 14:22:58 +01:00
Clint 512591533c add logicmonitor provider (#16314) 2017-10-11 13:00:42 -05:00
Martin Atkins 22101d6ec3 Update CHANGELOG.md 2017-10-10 11:57:51 -07:00
Gauthier Wallet ed9ba576e3 core: New interpolation function "chunklist"
This turns a list into a list of lists with each element (apart from possibly the last) being the given length.
2017-10-10 11:56:13 -07:00
James Bardin 935890ec64 Merge pull request #16294 from doertedev/master
Aliasing the module to omit errors thrown at build time
2017-10-09 21:27:48 -04:00
Stefan Staudenmeyer 14792035cf Aliasing the module to omit errors thrown at build time
Signed-off-by: Stefan Staudenmeyer <stefan.staudenmeyer@instana.com>
2017-10-09 16:44:58 +02:00
James Bardin 611b1ced59 retry on s3 state upload errors
While #16243 added the ability to retry getting a state from S3, Put can
return the same InternalError status. Use the same retry logic when
uploading state to S3.
2017-10-08 16:46:33 -04:00
James Bardin f5e9a20c66 reset testLockHook 2017-10-08 16:24:45 -04:00
James Bardin 25a8227291 add broken test for lock lost on connection error
Add a way to inject network errors by setting an immediate deadline on
open consul connections. The consul client currently doesn't retry on
some errors, and will force us to lose our lock.

Once the consul api client is fixed, this test will fail.
2017-10-08 16:16:57 -04:00
James Bardin fd9adcdb36 only init one consul client, and lower keepalive
The consul Client is analogous to an http.Client, and we really don't
need more than 1. Configure a single client and store it in the backend.

Replace the default Transport's Dialer to reduce the KeepAlive setting
from 30s to 17s. This avoids racing with the common network timeout
value of 30s, and is also coprime to other common intervals.
2017-10-08 11:51:35 -04:00
James Bardin d0ecb232ae record consul session ID in lock info
This can help correlate TF and consul logs
2017-10-08 11:24:43 -04:00
Martin Atkins 7916268d7f build: use Go 1.9.1 within Travis-CI unit test runs 2017-10-06 16:20:19 -07:00
Martin Atkins 5cd00a13ec command: use new diagnostics output for config errors
This uses the new diagnostics printer for config-related errors in the
main five commands that deal with config.

The immediate motivation for this is to allow HCL2-produced diagnostics
to be printed out in their full fidelity, though it also slightly changes
the presentation of other errors so that they are not presented in all
red text, which can be hard to read on some terminals.
2017-10-06 11:46:07 -07:00
Martin Atkins ea81e75a4e command: utility for rendering tfdiag diagnostics
This new method showDiagnostics takes any value that would be accepted by
tfdiags.Append and renders it to the UI.

This is intended to encourage consistent handling of the different kinds
of errors and diagnostics that can be produced, and allow richer error
objects like the HCL2 diagnostics to be easily unwrapped and shown in
their full-fidelity.
2017-10-06 11:46:07 -07:00
Martin Atkins 780e758f1e tfdiags: Allow construction of RPC-friendly Diagnostics
Due to the use of interfaces, Diagnostics is not super-friendly to the gob
encoding we currently use for plugin RPC. To mitigate this, we provide
a helper that converts all of the wrapped objects into a predictable flat
structure that we can pre-emptively register with gob.

This means that the decoded Diagnostics still has the same meaning as
the original, though the original wrapped errors (if any) are lost and
thus our errwrap integration won't be effective any longer.
2017-10-06 11:46:07 -07:00
Martin Atkins ab5efb805c tfdiags: SourceRange.StartString
This helper provides a concise string showing the filename, start line and
start column for a range, for easy inclusion in error messages.
2017-10-06 11:46:07 -07:00
Martin Atkins fc20f419dd config and command: use errwrap to propagate config load errors
Previously we were using fmt.Sprintf and thus forcing the stringification
of the wrapped error.

Using errwrap allows us to unpack the original error at the top of the
stack, which is useful when the wrapped error is really a hcl.Diagnostics
containing potentially-multiple errors and possibly warnings.
2017-10-06 11:46:07 -07:00