Commit Graph

20600 Commits

Author SHA1 Message Date
Martin Atkins 71e68f06c4 config: allow tests to force using the HCL2 loader
Currently the default for tests is to use the old HCL loader, but we need
to be able to test aspects of the new loader as we work through the
experimental phase. This new function testConfigHCL2 is the same as
testConfig except that it forces the use of HCL2 even if the opt-in
comment isn't present, thus allowing us to implement tests that ensure
that the exact same file works in both the old and new cases.

Once the HCL2 loader becomes the default this function will be removed
and callers will start calling into the normal testConfig function.
2017-10-03 17:47:01 -07:00
Martin Atkins d91327eaa0 config: allow HCL2 experiment opt-in (build-time flag to enable)
Use the new HCL2 config loader when the opt-in comment #terraform:hcl2 is
present in a .tf file.

For now this is disabled for "normal" builds and enabled only if
explicitly configured via a linker flag during build. This is because it's
not yet in a good state to be released: the HCL2 loader produces RawConfig
objects that the validator and interpolator can't yet deal with, and so
using HCL2 for anything non-trivial currently causes Terraform to crash
in real use.
2017-10-03 17:47:01 -07:00
Martin Atkins b0215fcd0f config: HCL2 config loader
This loader uses the HCL2 parser and decoder to process a config file,
and then transforms the result into the same shape as would be produced
by the HCL config loader.

To avoid making changes to the existing config structures (which are
depended on across much of the codebase) we first decode into a set of
HCL2-tailored structs and then process them into the public-facing structs
that a loader is expected to return. This is a compromise to keep the
config package API broadly unchanged for now. Once we're ready to remove
the old HCL loader (which implies that we're ready to support HCL2
natively elsewhere in the codebase) we will be able to simplify this
quite considerably.

Due to some mismatches of abstraction between HCL/HIL and HCL2, some
shimming is required to get the required result.
2017-10-03 17:47:01 -07:00
Martin Atkins edbbe41b44 config: allow a HCL2 body to piggy-back on a RawConfig
At this time we're not ready to refactor the various uses of RawConfig
in Terraform core, so we'll smuggle a HCL2 body within a degenerate
RawConfig object that we can then recognize and unpack once this object
is returned to us in an interpolation call.
2017-10-03 17:47:01 -07:00
Martin Atkins c202afeca9 govendor fetch github.com/hashicorp/hcl2/... 2017-10-03 17:47:01 -07:00
Martin Atkins 86343fa895 Remove test output file accidentally added in 52c97e9f
Also, fix the missing gitignore rule that caused this to be added in the
first place.
2017-10-03 17:32:26 -07:00
James Bardin c7e9e98500 update CHANGELOG.md 2017-10-03 14:31:46 -04:00
James Bardin 91442b7146 Merge pull request #15680 from brunomcustodio/etcdv3-backend
[WIP] etcd v3 backend with lock support.
2017-10-03 14:15:58 -04:00
James Bardin e34eb38672 Merge branch 'v0.10.7-branch' into stable-website 2017-10-03 13:38:29 -04:00
James Bardin 8cf0a8ca9c faster DAG transitive reduction
In the case of highly-connected graphs, the TransitiveReduction process
was far too computationally intensive. Since no operations are applied
to the nodes, and the walk order is not even user visible, we don't need
to sort them n^2 times.
2017-10-03 11:24:59 -04:00
James Bardin 7d5c320d0e Merge pull request #16080 from Banno/add-msg-on-ssh-errors
communicator/ssh: add what error details we can for the user
2017-10-03 10:21:57 -04:00
Adam Shannon c9c2823f62 communicator/ssh: add what error details we can for the user
ssh.Waitmsg's String() method provides output which can include the
process status, signal, and message
2017-10-03 09:06:02 -05:00
James Bardin d477d1f6d4 Merge pull request #15553 from bonifaido/custom_s3_backend
Allow non-AWS S3 backends
2017-10-02 19:39:48 -04:00
James Bardin 67be68b521 Merge pull request #16242 from hashicorp/jbardin/race
minor race fix in tests
2017-10-02 16:50:02 -04:00
James Bardin 08ac4f3a2e Merge pull request #16089 from wgliang/optimization-merge-slice
Optimization merge slice
2017-10-02 16:40:13 -04:00
James Bardin 3662aac93f Merge pull request #16204 from hashicorp/jbardin/outputs
make outputs error
2017-10-02 16:37:36 -04:00
James Bardin e0ee1e4d6e Merge pull request #16070 from octo/defaultdatadir
backend/local: Remove unused const DefaultDataDir.
2017-10-02 16:27:23 -04:00
James Bardin 3ed7b1abec fix the providerWithLocals test
This test was set to fail once this issue was fixed, and now it's fixed.
2017-10-02 16:21:37 -04:00
James Bardin 3c2b3df5a9 add some output error tests
Add some tests for output errors and catch the errors behind the output
errors feature flag.
2017-10-02 16:20:29 -04:00
James Bardin 715036d209 put output errors behind a feature flag
We're going to start merging breaking functgionality behind feature
flags, to reduce the need for long-lived feature branches.
2017-10-02 16:20:29 -04:00
James Bardin 35c6a4e89d add DestroyValueReferenceTransformer
DestroyValueReferenceTransformer is used during destroy to reverse the
edges for output and local values. Because destruction is going to
remove these from the state, nodes that depend on their value need to be
visited first.
2017-10-02 16:20:29 -04:00
James Bardin 3414e1b996 remove dead code 2017-10-02 16:20:29 -04:00
James Bardin 7fde7d9f20 make a destroy plan work the same as destroy
When working on an existing plan, the context always used walkApply,
even if the plan was for a full destroy. Mark in the plan if it was
icreated for a destroy, and transfer that to the context when reading
the plan.
2017-10-02 16:20:29 -04:00
James Bardin 78964d305c remove broken outputs from targeted graph
A Targeted graph may include outputs that were transitively included,
but if they are missing any dependencies they will fail to interpolate
later on.

Prune any outputs in the TargetsTransformer that have missing
dependencies, and are not depended on by any resource. This will
maintain the existing behavior of outputs failing silently ni most
cases, but allow errors to be surfaced where the output value is
required.
2017-10-02 16:20:29 -04:00
James Bardin a048bcffa0 continue on output errors during Input
Module outputs may not have complete information during Input, because
it happens before refresh. Continue process on output interpolation
errors during the Input walk.
2017-10-02 16:20:29 -04:00
James Bardin 1ad97f6be8 use an EvalOpFilter for module variables
Remove the Input flag threaded through the input graph creation process
to prevent interpolation failures on module variables.
Use an EvalOpFilter instead to inset the correct EvalNode during
walkInput. Remove the EvalTryInterpolate type, and use the same
ContinueOnErr flag as the output node for consistency and to try and
keep the number possible eval node types down.
2017-10-02 16:20:29 -04:00
James Bardin c4dd31e62b "destroy" outputs too
Ouptuts don't need to be re-evaluated during destroy, since everything
is already in the state, so we can simply remove them.
2017-10-02 16:20:29 -04:00
James Bardin fc9ada463c minor race fix in tests 2017-10-02 16:19:01 -04:00
James Bardin 71e47e56a0 release: clean up after v0.10.7 2017-10-02 18:54:33 +00:00
James Bardin 8ba8fc79c1
v0.10.7 2017-10-02 18:46:49 +00:00
James Bardin 5f9bf20ba6 update CHANGELOG.md 2017-10-02 14:39:47 -04:00
James Bardin d367afe6d4 Merge pull request #16233 from alex1x/master
Fixes #16226 - two changes to the Provisioners sidebar on the website
2017-10-02 13:44:58 -04:00
Alex Tasioulis 4e6289dad0 website: change order of provisioners in sidebar list
Assume null_resource should also be ordered alphabetically
2017-10-01 22:03:39 +01:00
Alex Tasioulis 825a098c56 website: add salt-masterless provisioner to sidebar
Fixes #16226
2017-10-01 22:02:43 +01:00
James Bardin cef16676f7 Merge pull request #16219 from hashicorp/jbardin/local-destroy
Improve the test to show a failure when using locals in a provider
2017-09-29 17:36:04 -04:00
James Bardin 4694f88896 Merge pull request #16161 from hashicorp/jbardin/go-getter
update go-getter
2017-09-29 17:24:36 -04:00
James Bardin da0e74276a fix provider with local value test and docs
Make sure this fails during destroy

Add a note in the documentation that local values may fail during
destroy.
2017-09-29 17:14:07 -04:00
Martin Atkins 4e04e4cfdd Update CHANGELOG.md 2017-09-29 14:06:13 -07:00
Martin Atkins 34956cd124 website: Document the plugin cache
Since we don't currently auto-install provisioner plugins this is
currently placed on the providers documentation page and referred to as
the "Provider Plugin Cache". In future this mechanism may also apply to
provisioners, in which case we'll figure out at that point where better
to place this information so it can be referenced from both the provider
and provisioner documentation pages.
2017-09-29 14:03:09 -07:00
Martin Atkins f19bf25945 website: don't recommend "providers" block in CLI config
This mechanism for configuring plugins is now deprecated, since it's not
capable of declaring plugin versions. Instead, we recommend just placing
plugins into a particular directory, which is now documented on the
main providers documentation page and linked from the more detailed docs
on plugins in general.
2017-09-29 14:03:09 -07:00
Martin Atkins adbb9d0cf9 website: checkpoint settings refer to CLI config file docs
Previously we described inline here where to put the .terraformrc file,
but now we have a separate page all about this file that gives us more
room to describe in more detail where the file is placed and what else it
can do.
2017-09-29 14:03:09 -07:00
Martin Atkins 35959b8689 website: document explicitly the CLI configuration file
Previously this file was mentioned in passing in a few locations. It now
has enough functionality to warrant its own page.
2017-09-29 14:03:09 -07:00
Martin Atkins 30e6f4e066 command/e2etest: test installation of plugins from cache
This is a tough one to unit tests because the behavior is tangled up in
the code that hits releases.hashicorp.com, so we'll add this e2etest as
some extra insurance that this works end-to-end.
2017-09-29 14:03:09 -07:00
Martin Atkins cb17a9a607 main: allow enabling plugin caching via config file or environment
Either the environment variable TF_PLUGIN_CACHE_DIR or a setting in the
CLI config file (~/.terraformrc on Unix) allow opting in to the plugin
caching behavior.

This is opt-in because for new users we don't want to pollute their system
with extra directories they don't know about. By opting in to caching, the
user is assuming the responsibility to occasionally prune the cache over
time as older plugins become stale and unused.
2017-09-29 14:03:09 -07:00
Martin Atkins 5c0670fdf4 command: allow callers to activate a plugin cache
Here we add a new caller-settable field to command.Meta that activates
a read-through cache directory for plugin installation.
2017-09-29 14:03:09 -07:00
Martin Atkins 879899d434 plugin/discovery: plugin caching mechanism
For users that have metered or slow internet connections it is annoying
to have Terraform constantly re-downloading the same files when they
initialize many separate directories.

To help such users, here we add an opt-in mechanism to use a local
directory as a read-through cache. When enabled, any plugin download will
be skipped if a suitable file already exists in the cache directory. If
the desired plugin isn't in the cache, it will be downloaded into the
cache for use next time.

This mechanism also serves to reduce total disk usage by allowing
plugin files to be shared between many configurations, as long as the
target system isn't Windows and supports either hardlinks or symlinks.
2017-09-29 14:03:09 -07:00
Martin Atkins 12d6bc8c30 plugin/discovery: ignore non-files when discovering
If we encounter something that isn't a file -- for example, a dangling
symlink whose referent has been deleted -- we'll ignore it so that we
can either later produce a "no such plugin" error or auto-install a plugin
that will actually work.
2017-09-29 14:03:09 -07:00
Martin Atkins 3f401f0cd4 main: make configuration available when initializing commands
This, in principle, allows us to make use of configuration information
when we populate the Meta structure, though we won't actually make use
of that until a subsequent commit.
2017-09-29 14:03:09 -07:00
James Bardin 60d8b6c4d7 Merge pull request #16214 from hashicorp/jbardin/registry-url
remove extra slash from registry urls
2017-09-28 17:52:34 -04:00
Martin Atkins f45aae9b70 build: run end-to-end tests during Travis run
We don't usually run "acceptance tests" during a Travis run, but this
particular suite doesn't require any special credentials since it just
accesses releases.hashicorp.com to download plugins, so therefore it's
safe to run in Travis at the expense of adding a few more seconds to
the runtime.

Running it in Travis can therefore give us some extra confidence for
pull requests that may inadvertently break certain details of the
workflow, as well as ensuring that these tests are kept up-to-date as
the system changes.
2017-09-28 14:35:51 -07:00