Commit Graph

1566 Commits

Author SHA1 Message Date
Radek Simko 953eae7e4b
command/format: Fix rendering of different types 2019-01-23 13:13:48 +00:00
Radek Simko f3d1565d6f
command/format: Fix tuple diff formatting 2019-01-23 11:17:55 +00:00
Radek Simko 0dff8fe5e0
Add failing test case for tuple 2019-01-22 16:49:49 +00:00
Radek Simko 3d0a25c65d
command/format: Fix nested (JSON) object formatting 2019-01-22 16:26:28 +00:00
Radek Simko c5ba7469be
command/format: Fix rendering of unknown elements in set/map/list 2019-01-21 15:05:59 +00:00
Radek Simko 98cc99e632
command/format: Add crashing test with unknown element in map 2019-01-21 15:04:13 +00:00
Radek Simko 73225c7aeb
command/format: Add crashing test with unknown element in list 2019-01-21 15:04:13 +00:00
Radek Simko e7e8b7358f
command/format: Add crashing test with unknown element in set 2019-01-21 15:04:13 +00:00
Martin Atkins 8b094f48f7 command: Update "terraform get" to use the new module installer
We missed this on the initial update pass because this was calling
directly into the module package API rather than going through the Meta
methods that we updated for the new config loader.

m.installModules here is the same method that "terraform init" is using
for this purpose, ensuring the two will behave the same way. This changes
the output a little compared to the old installer, but it still includes
the important information about where each module is coming from.
2019-01-17 16:52:34 -08:00
Martin Atkins 176a5abfd3 command: Restore single-file support in "terraform fmt"
This possibility was lost in the rewrite to use HCL2, but it's used by
a number of external utilities and text editor integrations, so we'll
restore it here.

Using the stdin/stdout mode is generally preferable for text editor use
since it allows formatting of the in-memory buffer rather than directly
the file on disk, but for editors that don't have support for that sort of
tooling it can be convenient to just launch a single command and directly
modify the on-disk file.
2019-01-17 14:21:18 -08:00
Martin Atkins b0a43cab84 command: "terraform fmt" must fail if file has invalid syntax
Since the HCL formatter only works with tokens, it can in principle be
called with any input and produce some output. However, when given invalid
syntax it will tend to produce nonsensical results that may drastically
change the input file and be hard for the user to undo.

Since there's no strong reason to try to format an invalid or incomplete
file, we'll instead try parsing first and fail if parsing does not
complete successfully.

Since we talk directly to the HCL API here this is only a _syntax_ check,
and so it can be applied to files that are invalid in other ways as far
as Terraform is concerned, such as using unsupported top-level block types,
resource types that don't exist, etc.
2019-01-17 14:21:18 -08:00
findkim 7816e61614 Bump installer protocol version to 5 and separate client and server protocol references 2019-01-16 15:07:57 -06:00
Radek Simko 13896d72c5
command/format: Render empty object as {} 2019-01-15 14:34:49 +00:00
Radek Simko 5f14b7a7f2
command/format: Render empty primitive list/set as [] 2019-01-15 14:34:49 +00:00
Radek Simko 09d19ca9d9
command/format: Render empty JSON object as {} 2019-01-15 14:34:48 +00:00
Radek Simko 0dd2d56f18
command/format: Render empty maps as {} 2019-01-15 14:34:46 +00:00
Kim Ngo 41a2376915
Merge pull request #19981 from findkim/colorize-protocol-mismatch-msg
Condense protocol mismatch UI error message
2019-01-14 15:12:09 -06:00
findkim 6e0de3e3f5 Nest incompatible provider protocol error to include dynamic custom msg 2019-01-14 14:56:47 -06:00
Radek Simko aae87c7921
Merge pull request #19990 from hashicorp/f-cmd-fmt-ignore-empty-str
command/format: Ignore removal of empty strings
2019-01-14 20:33:43 +00:00
Martin Atkins 86c02d5c35 command: "terraform init" can partially initialize for 0.12upgrade
There are a few constructs from 0.11 and prior that cause 0.12 parsing to
fail altogether, which previously created a chicken/egg problem because
we need to install the providers in order to run "terraform 0.12upgrade"
and thus fix the problem.

This changes "terraform init" to use the new "early configuration" loader
for module and provider installation. This is built on the more permissive
parser in the terraform-config-inspect package, and so it allows us to
read out the top-level blocks from the configuration while accepting
legacy HCL syntax.

In the long run this will let us do version compatibility detection before
attempting a "real" config load, giving us better error messages for any
future syntax additions, but in the short term the key thing is that it
allows us to install the dependencies even if the configuration isn't
fully valid.

Because backend init still requires full configuration, this introduces a
new mode of terraform init where it detects heuristically if it seems like
we need to do a configuration upgrade and does a partial init if so,
before finally directing the user to run "terraform 0.12upgrade" before
running any other commands.

The heuristic here is based on two assumptions:
- If the "early" loader finds no errors but the normal loader does, the
  configuration is likely to be valid for Terraform 0.11 but not 0.12.
- If there's already a version constraint in the configuration that
  excludes Terraform versions prior to v0.12 then the configuration is
  probably _already_ upgraded and so it's just a normal syntax error,
  even if the early loader didn't detect it.

Once the upgrade process is removed in 0.13.0 (users will be required to
go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can
be simplified to remove that special mode, but the idea of doing the
dependency version checks against the liberal parser will remain valuable
to increase our chances of reporting version-based incompatibilities
rather than syntax errors as we add new features in future.
2019-01-14 11:33:21 -08:00
Martin Atkins 0c0a437bcb Move module install functionality over to internal/initwd 2019-01-14 11:33:21 -08:00
Radek Simko d96f4fa77b
command/format: Ignore removal of empty strings 2019-01-13 22:56:04 +00:00
Kristin Laemmert 5df9cd0f52
command/show: tests for -json output (#19980)
* command/show: added test scaffold for json output

More test cases will be added once the basic shape of the tests is
validated.

- command/json* packages now sort resources by address, matching
behavior elsewhere
- using cmp in tests instead of reflect.DeepEqual for the diffs
- updating expected output in tests to match sorting
2019-01-11 15:13:55 -08:00
Radek Simko bc4b7cad68
command/format: Render null in dark gray (#19616) 2019-01-11 19:27:09 +00:00
Kristin Laemmert cdf7cc2449
command/json*: updating documentation and adding tests (#19944)
A few minor fixes and cleanups as a result of said tests. Hooray for
eventual consistency!
2019-01-09 08:59:11 -08:00
Sander van Harmelen fb57e9b26f command/state: add proper locking 2019-01-08 14:57:52 +01:00
Kristin Laemmert 3e9c51c726
command/jsonstate: marshal state into a machine readable blob o'json (#19911)
* command/jsonstate: marshal state into a machine readable blob o'json
* consistent json tags are nice
2019-01-03 12:08:03 -08:00
Kristin Laemmert 2b6dc13f29
jsonplan: remove "proposed_unknown" structure in favor of "after_unknown" field in "change" (#19709)
* jsonplan: remove "proposed_unknown" structure in favor of
"after_unknown" field in "change"
2018-12-20 14:30:18 -08:00
Martin Atkins 2cf63d068f command: Always normalize config path before operations
Previously we were doing this rather inconsistently: some commands would
do it and others would not. By doing it here we ensure we always apply the
same normalization, regardless of which operation we're running.

This normalization is mostly for cosmetic purposes in error messages, but
it also ends up being used to populate path.module and path.root and so
it's important that we always produce consistent results here so that
we don't produce flappy changes as users work with different commands.

The fact that thus mutates a data structure as a side-effect is not ideal
but this is the best place to ensure it always gets applied without doing
any significant refactoring, since everything after this point happens in
the backend package where the normalizePath method is not available.
2018-12-19 13:47:42 -08:00
Kristin Laemmert 126e5f337f
json output of terraform plan (#19687)
* command/show: adding functions to aid refactoring

The planfile -> statefile -> state logic path was getting hard to follow
with blurry human eyes. The getPlan... and getState... functions were
added to help streamline the logic flow. Continued refactoring may follow.

* command/show: use ctx.Config() instead of a config snapshot

As originally written, the jsonconfig marshaller was getting an error
when loading configs that included one or more modules. It's not clear
if that was an error in the function call or in the configloader itself,
  but as a simpler solution existed I did not dig too far.

* command/jsonplan: implement jsonplan.Marshal

Split the `config` portion into a discrete package to aid in naming
sanity (so we could have for example jsonconfig.Resource instead of
jsonplan.ConfigResource) and to enable marshaling the config on it's
own.
2018-12-19 11:08:25 -08:00
James Bardin 149ccd929e missing commits from 19688 2018-12-18 16:54:09 -08:00
James Bardin 9667e06a03 decode backend hash as uint64
Older versions of terraform could save the backend hash number in a
value larger than an int.

While we could conditionally decode the state into an intermediary data
structure for upgrade, or detect the specific decode error and modify
the json, it seems simpler to just decode into the most flexible value
for now, which is a uint64.
2018-12-18 17:57:44 -05:00
Charles Kenney 2dfc3a399d command: allow -no-color option on "providers" command 2018-12-17 10:55:16 -08:00
Radek Simko f64978b64c
backend/local: Render CBD replacement (+/-) correctly (#19642)
* backend/local: Render CBD replacement (+/-) correctly

* command/format: Use IsReplace helper function
2018-12-14 13:45:47 +00:00
Masayuki Morita 33e1839730 command/hook_ui: Truncate the ID considering multibyte characters
Fixes #18822

The `tuncatedId` function had been introduced in #12261 and increased the
`maxIdLen` to 80 in #13317. Since the number of bytes itself seems to be
unimportant, the ID should be truncated to 80 characters, not 80 bytes.
2018-12-13 11:18:14 +09:00
Radek Simko b28efa0bf3
command/format: Fix tests 2018-12-12 14:28:12 +00:00
Radek Simko 0b981fa641
command/format: Fix rendering of force-new updates 2018-12-12 14:19:17 +00:00
Radek Simko fe117e9f02
command/format: Fix rendering of nested blocks during update 2018-12-11 18:07:21 +00:00
Radek Simko 2df886397b
command/format: Add more tests to cover non-primitive fields 2018-12-11 14:16:25 +00:00
Radek Simko e3e459a8d4
command/format: Restructure tests 2018-12-11 14:16:17 +00:00
Radek Simko 49e7026bdd
command/format: Add tests for ResourceChange renderer 2018-12-10 17:42:45 +00:00
Kristin Laemmert e3cecb7d56
format/state: added missing newline in the `outputs` output (#19542) 2018-12-04 11:37:22 -08:00
Radek Simko 9e5677adeb
Merge pull request #19453 from hashicorp/b-whole-body-diag-rendering
command/format: Fix rendering of attribute-agnostic diagnostics
2018-11-26 23:52:21 +00:00
Radek Simko 501fcd1e21
Fix tests after upgrading hcl 2018-11-26 23:38:37 +00:00
Radek Simko 4f2868a019
command/format: Fix rendering of attribute-agnostic diagnostics 2018-11-26 23:38:37 +00:00
Sander van Harmelen ef9054562e commands: make sure the correct flagset is used
A lot of commands used `c.Meta.flagSet()` to create the initial flagset for the command, while quite a few of them didn’t actually use or support the flags that are then added.

So I updated a few commands to use `flag.NewFlagSet()` instead to only add the flags that are actually needed/supported.

Additionally this prevents a few commands from using locking while they actually don’t need locking (as locking is enabled as a default in `c.Meta.flagSet()`.
2018-11-23 16:13:34 +01:00
Sander van Harmelen 79a9a15879 command/state: lock when pushing state
Next to adding the locking for the `state push` command, this commit also fixes a small bug where the lock would not be propertly released when running the `state show` command.

And finally it renames some variables in the `[un]taint` code in order to try to standardize the var names of a few frequently used variables (e.g. statemgr.Full, states.State, states.SyncState).
2018-11-20 11:15:16 +01:00
Martin Atkins 884aa387b8 command: Use vendoring when building helper programs in tests
In a couple places in tests we execute a child "go build" to make a helper
program. Now that we're running in module mode, "go build" will normally
default to downloading and caching dependencies, which we don't want
because we're still using vendoring for the moment.

Therefore we need to instruct these child builds to use vendoring too,
avoiding the need to download all of the dependencies and ensuring that
we'll be building with the same dependencies that we'd use for a normal
build.
2018-11-19 11:41:52 -08:00
Martin Atkins 5255e85238 "go fmt" fixups
Apparently my editor is still not reliably formatting on save, so I missed
a few formatting quirks in these files.
2018-11-19 09:02:35 -08:00
Martin Atkins 73c9521a04 command/e2etest: Temporarily disable tests that access network
Several of these tests rely on external services (e.g. Terraform Registry)
that have not yet been updated to support the needs of Terraform v0.12.0,
so for now we'll skip all of these tests and wait until those systems have
been updated.

This should be removed before Terraform v0.12.0 final to enable these
tests to be used as part of pre-release smoke testing.
2018-11-19 09:02:35 -08:00
Martin Atkins 37bc187f95 command: "terraform output" mustn't panic when no state is present
This is verified by TestOutput_noArgs.
2018-11-19 09:02:35 -08:00
Martin Atkins 53b5b95ef5 command: Fix TestRefresh_backup
The local filesystem state manager no longer creates backup files eagerly,
instead creating them only if on first write there is already a snapshot
present in the target file.

Therefore for this test to exercise the codepaths it intends to we must
create an initial state snapshot for it to overwrite, creating the backup
in the process.

There are several other tests for this behavior elsewhere, so this test
is primarily to verify that the refresh command is configuring the backend
appropriately to get the backups written in the desired location.
2018-11-19 09:02:35 -08:00
Martin Atkins 762a173c7f command: Fix TestRefresh_outPath
We now only create a backup state file if the given output file already
exists, which it does not in this test.

(The behavior of creating the backup files is already covered by other
tests, so no need for this one go out of its way to do it.)
2018-11-19 09:02:35 -08:00
Martin Atkins e2ba90fdfa command: Fix TestMetaBackend_planLocalMatch
We now don't create a local state backup until the first snapshot write,
so we don't expect there to be a backup file until the end of the test.
(There is already a check at the end there, unmodified by this change.)
2018-11-19 09:02:35 -08:00
Martin Atkins 48601d261d states/statemgr: In Filesystem, back up output file, not input file
The filesystem backend has the option of using a different file for its
initial read.

Previously we were incorrectly writing the contents of that file out into
the backup file, rather than the prior contents of the output file. Now
we will always read the output file in RefreshState in order to decide
what we will back up but then we will optionally additionally read the
input file and prefer its content as the "current" state snapshot.

This is verified by command.TestMetaBackend_planLocalStatePath and
TestMetaBackend_configureNew, which are both now passing.
2018-11-19 09:02:35 -08:00
Martin Atkins 27abd9c6b8 command: Fix TestMetaBackend_localDoesNotDeleteLocal
The changes to how we handle setting the state path on the local backend
broke the heuristic we were using here for detecting migration from one
local backend to another with the same state path, which would by default
end up deleting the state altogether after migration.

We now use the StatePaths method to do this, which takes into account
both the default values and any settings that have been set.

Additionally this addresses a flaw in the old method which could
potentially have deleted all non-default workspace state files if the
"path" setting were changed without also changing the "workspace_dir"
setting. This new approach is conservative because it will preserve all
of the files if any one overlaps.
2018-11-19 09:02:35 -08:00
Martin Atkins be79bf0412 command: Fix TestPlan_outBackend
In an earlier change we fixed the "backendFromConfig" codepath to be
able to properly detect changes to the -backend-config arguments during
"terraform init", but this detection is too strict for the normal case
of running an operation in a previously-initialized directory.

Before any of the recent changes, the logic here was to selectively update
the hash to include -backend-config settings in the init case. Since
that late hash recalculation was confusing, here we take the alternative
path of using the hash only in the normal case and full value comparison
in the init case. Treating both of these cases separately makes things
marginally easier to follow here.
2018-11-19 09:02:35 -08:00
Martin Atkins b316e4ab56 command: Fix TestImport_remoteState
The import command was imposing the default state path at the CLI level,
rather than leaving that to be handled by the backend. As a result, the
output state was always forced to be terraform.tfstate, regardless of
the backend settings.
2018-11-19 09:02:35 -08:00
Martin Atkins 2b9f92be31 command: Partially fix TestMetaBackend_planLocalStatePath
This test is testing some strange implementation details of the old
local backend which do not hold with the new filesystem state manager.
Specifically, it was expecting state to be read from the stateOutPath
rather than the statePath, which makes no sense here because the backend
is configured to read from the default terraform.tfstate file (which does
not exist.)

There is another problem with this test which will be addressed in a
subsequent commit.
2018-11-19 09:02:35 -08:00
Martin Atkins aecb66d3db command: Fix TestMetaBackend_configuredChangeCopy_multiToNoDefaultWithoutDefault
As part of integrating the new "remote" backend we relaxed the requirement
that a "default" workspace must exist in all backends and now skip
migrating empty workspace states to avoid creating unnecessary "default"
workspaces when switching between backends that require it and backends
that don't, such as when switching from the local backend (which always
has a "default" workspace) to Terraform Enterprise.
2018-11-19 09:02:35 -08:00
Martin Atkins ec27526cc3 command: Fix TestMetaBackend_configuredChangeCopy_multiToMulti
This was failing because we now handle the settings for the local backend
a little differently as a result of decoding it with the HCL2 machinery.

Specifically, the backend.State* fields are now assumed to be what is
given in configuration, and any CLI overrides are maintained separately
in OverrideState* fields so that they can be imposed "just in time" in
StatePaths.

This is particularly important because OverrideStatePath (when set) is
used regardless of workspace name, while StatePath is a suitable value
only for the "default" workspace, with others needing to be constructed
from StateWorkspaceDir instead.
2018-11-19 09:02:35 -08:00
Martin Atkins 6c7cecfbd8 command: More logging during migration
This just finishes off the logging added in earlier commits to get all
the way through to the actual migration call.
2018-11-19 09:02:35 -08:00
Martin Atkins f39a5d0962 command: Fix various TestMetaBackend-prefix tests
Our new state model has a different implementation of "empty" that doesn't
consider lineage/serial, so we need to have some actual content in these
state fixtures to avoid them being skipped during state migrations.
2018-11-19 09:02:35 -08:00
Martin Atkins 22c84c71a4 command: Use statemgr.Import and statemgr.Export for state push and pull
We previously hacked around the import/export functionality being missing
in the statemgr layer after refactoring, but now it's been reintroduced
to fix functionality elsewhere we should use the centralized Import and
Export functions to ensure consistent behavior.

In particular, this pushes the logic for checking lineage and serial
during push down into the state manager itself, which is better because
all other details about lineage and serial are managed within the state
managers.
2018-11-19 09:02:35 -08:00
Martin Atkins aacbe1d14b command: Fix TestMetaBackend_configureNewWithState
This test was initially failing because its fixture had a state which our
new state models consider to be "empty", and thus it was not migrated.

After fixing that (by adding an output to the fixture), this revealed a
bug that the lineage was not being persisted through the migration. This
is fixed by using the statemgr.Migrate method instead of writing via the
normal Writer interface, which allows two cooperating state managers to
properly transfer the lineage and serial along with the state snapshot.
2018-11-19 09:02:35 -08:00
Martin Atkins 24046ab833 command: More TRACE logging for the Backend instantiation codepaths 2018-11-19 09:02:35 -08:00
Martin Atkins 2293391241 command: Fix TestMetaBackend_emptyWithExplicitState
This test was incorrectly updated in a previous iteration, with it
creating a modified state to write but then not actually writing it,
writing an empty test state instead.

This made the test fail because a backup state file is created only if
the new state snapshot is different to the old when written.
2018-11-19 09:02:35 -08:00
Martin Atkins 8c54da0ad2 command: TestInit_fromModule_explicitDest guard against other tests
Some other test is leaving behind a terraform.tfstate after it concludes,
which can cause this test to fail in a strange way due to picking up
extra provider requirements from that state.

This check doesn't fix that problem, but it at least makes the test fail
in a more helpful way to avoid time wasted trying to debug this test when
it's some other test that actually has the bug.
2018-11-12 15:19:55 -08:00
Martin Atkins ced06a4ca3 command: fix panic in TestStatePush_lineageMismatch
This test is currently failing due to the command completing successfully,
which would previously cause a panic because we didn't properly initialize
the MockUi and so its error buffer is nil unless written to.

(The failure this was masking will be fixed in a subsequent commit.)
2018-11-12 15:19:55 -08:00
Martin Atkins 544c2932ce command: Fix TestInit_checkRequiredVersion
In prior refactoring we lost the required core version check from
"terraform init", which we restore here.

Additionally, this test used to have an incorrect name that suggested it
was testing something in the "getProvider" codepath, but version checking
happens regardless of what other options are selected.
2018-11-12 15:19:55 -08:00
Martin Atkins 9ba399bca8 command: Fix TestInit_getProvider
After all of the refactoring we were no longer checking the Terraform
version field in a state file, causing this test to fail.

This restores that check, though with a slightly different error message.
2018-11-12 15:19:55 -08:00
Martin Atkins f6d468ffd5 command: Fix TestInit_inputFalse
This test was using old-style state files as its input, differing only by
lineage. Since lineages are now managed within the state manager itself,
the test can't use that to distinguish the two files and so we put a
different output in each one instead.

This also introduces some TRACE logging to the migration codepaths.
There's some hard-to-follow control flow here and so this extra logging
helps to understand the reason for a particular outcome, and since this
codepath is visited only in "terraform init" anyway it doesn't hurt to
be a bit more verbose here.
2018-11-12 15:19:55 -08:00
Martin Atkins c0b7f58143 command: Fix detection of necessary backend migration
In the refactoring for new HCL this codepath stopped taking into account
changes to the CLI -backend-config options when deciding if a backend
migration is required.

This restores that behavior in a different way than it used to be: rather
than re-hashing the merged config and comparing the hashes, we instead
just compare directly the configuration values, which must be exactly
equal in order to skip migration.

This change is covered by the test TestInit_inputFalse, although as of
this commit it is still not passing due a downstream problem within the
migration code itself.
2018-11-12 15:19:55 -08:00
Sander van Harmelen 5e9414bf03 backend/migrations: only select workspaces if supported
If the backend does not support workspaces, we don't have to try to
select a workspace and we should not return an error.
2018-11-12 16:51:32 +01:00
Martin Atkins 297b3b8830 command: Fix TestInit_backendReinitConfigToExtra
This test was re-using the same InitCommand value to run multiple times,
which is not realistic. Since we now cache configuration source code
inside command.Meta on load, it's important that we use a fresh
InitCommand instance here so it'll see the modified configuration file
we've left on disk.
2018-11-09 09:48:03 -08:00
Martin Atkins b5547f00f6 command: backendConfig must mutate its copy, not the original
Here we were going to the trouble of copying the body so we could mutate
it, but then ended up mutating the original anyway and then returning the
unmodified copy. Whoops!

This fix is verified by a number of "init" command tests that exercise the
-backend-config option, including TestInit_backendConfigFile and several
others whose names have the prefix TestInit_backendConfig .
2018-11-09 09:48:03 -08:00
Martin Atkins 8b603e4877 command: Name the Terraform Registry specifically in error message
When we originally wrote this message we struggled a bit for how to refer
to the releases server without writing an awkwardly-ungrammatical
sentence, and so "the official repository" became a placeholder name for
it.

Now that we'll be looking in Terraform Registry this gives us a nice
proper noun to use. This message will need to evolve more as our
integration with the registry gets more sophisticated, but for now this
works.
2018-11-09 09:48:03 -08:00
Martin Atkins 592850e22c command: Fix TestApply_plan_remoteState
Some over-zealous bulk updating of this test file caused this test to be
producing a remote state config cache file on disk when it doesn't
actually need one: the backend config comes from the plan file when
applying a saved plan.
2018-11-09 09:48:03 -08:00
Martin Atkins c05a556c0e command: Fix TestRefresh_backup
This test intentionally overrides the backup file location using the
-backup option, so the backup file is not in the default location for this
one.
2018-11-09 09:48:03 -08:00
Martin Atkins ad276142b4 command: Fix the command package test build
Some merging conflict shenanigans here led to this usage not lining up
with the imported symbol name, meaning that the tests couldn't compile any
more.
2018-11-08 14:39:24 -08:00
Martin Atkins 1e45d30036 command: Fix TestPlan_outBackend
We missed fixing this up during the big updates for the new plan/state
models since the failures were being masked by testBackendState being
broken.

This is the same sort of update made to many other tests: add schema to
the mock provider, adjust for the new plan/state types, and make
allowances for the new built-in diffing behavior in core.
2018-11-08 08:57:11 -08:00
Martin Atkins c3d11b762b command: Fix testBackendState
The hashing function for cached backend configuration is different now, so
our hard-coded hash of the HTTP backend address wasn't working anymore.

Here we update the hash so that tests using this test backend will work
again. Rather than leaving it hard-coded, we'll instead compute it the
same way as "terraform init" would.

In practice only one test is actually using this function right now, so
we also update the test fixture for that test (TestPlan_outBackend) to
match the new expectations, though as of this commit it's still failing
with an unrelated error.
2018-11-08 08:57:11 -08:00
Martin Atkins e20346bf4f command: fix TestMeta_process
The mission of this process method used to include dealing with
auto-loaded tfvars files, but it doesn't do that anymore.

It does still deal with the -no-color option, but the test wasn't
exercising that part before.

Now the test here focuses on the -no-color behavior.

The process method still has a "vars" flag argument which is no longer
used. Since this is an unexported method we could potentially address this
but this commit is intentionally limited only to fixing the test.
2018-11-08 08:57:11 -08:00
Martin Atkins fcf3f643ce command: Fix TestPlan_shutdown
Comments here indicate that this was erroneously returning an error but
we accepted it anyway to get the tests passing again after other work.
The tests over in the "terraform" package agree that cancelling should be
a successful outcome rather than an error.

I think that cancelling _should_ actually be an error, since Terraform did
not complete the operation it set out to complete, but that's a change
we'd need to make cautiously since automation wrapper scripts may be
depending on the success-on-cancel behavior.

Therefore this just fixes the command package test to agree with the
Terraform package tests and adds some FIXME notes to capture the potential
that we might want to update this later.
2018-11-08 08:57:11 -08:00
Martin Atkins 0ea8aa6fe5 command: Fix TestWorkspace_createWithState
The State.Equal function is now more precise than this test needs. It's
only trying to distinguish between an empty state and a non-empty state,
so the string representation of state is good enough to get that done
while disregarding other subtle differences.
2018-11-08 08:57:11 -08:00
Sander van Harmelen 52a1b22f7a Implement the remote enhanced backend
This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.
2018-11-06 16:29:46 +01:00
Sander van Harmelen b62a22ab62 Add a VariableSourceType for names .tfvars files
This new source type should be used for variables loaded from .tfvars files that were explicitly passed as command line arguments (e.g. -var-file=foo.tfvars)
2018-11-05 19:29:34 +01:00
Sander van Harmelen 178ec8f7b4 Remove support for the -module-depth flag
# Conflicts:
#	backend/backend.go
2018-11-02 18:44:04 +01:00
Sander van Harmelen 643fccc923
Merge pull request #19250 from hashicorp/f-inputs
Fix the ability to ask for and save user input
2018-11-01 20:14:52 +01:00
Sander van Harmelen 5944e8e34f Fix the ability to ask for and save user input 2018-11-01 20:00:08 +01:00
Sander van Harmelen 5458a91985 command/state: update and fix the state show command 2018-10-27 15:15:25 +02:00
Sander van Harmelen 7ec3f96e3a command/state: update and fix the state mv command 2018-10-27 15:01:07 +02:00
Sander van Harmelen 19c1241a50 command/state: update and fix the state rm command 2018-10-24 10:59:33 +02:00
James Bardin e93d69f18b more nil/known checks before val.LengthInt 2018-10-19 16:51:15 -04:00
Sander van Harmelen 536c2fe6f1 Make `state mv` use the new `states.Filter` 2018-10-19 19:19:49 +02:00
Sander van Harmelen 5e11de460a
Merge pull request #19130 from hashicorp/f-state-push-pull
command/state: update and fix the state push and pull
2018-10-19 19:16:00 +02:00
Sander van Harmelen 7fbd93b5cd command/state: update and fix the state push and pull 2018-10-19 19:12:23 +02:00
Sander van Harmelen a88ba31b63
Merge pull request #19129 from hashicorp/f-show-list
command/state: update and fix the state list command
2018-10-19 19:10:09 +02:00
Sander van Harmelen 83168ea25f
Merge pull request #19127 from hashicorp/b-prepare-provider
Make the test compile again
2018-10-19 19:05:18 +02:00
Sander van Harmelen af1a471a05 command/state: update and fix the state list command 2018-10-19 16:31:12 +02:00
Sander van Harmelen 4d6626b3a8 Terraform show didn’t show absolute paths
Without using absolute paths any module info is lost in the output. And the attributes were randomly ordered and so changed between different executions of the command.
2018-10-19 15:38:11 +02:00
Sander van Harmelen b846c5f653 Make the test compile again
Not sure if these checks still make sense, but without this change the test don’t compile for the related packages.
2018-10-19 15:29:14 +02:00
Martin Atkins 17b883f592 command/format: Include variable values in diagnostic messages
When HCL encounters an error during expression evaluation, it annotates
its diagnostics with information about the expression that was being
evaluated and the EvalContext it was evaluated in.

This gives us enough information to show helpful hints to the user about
the final values of any reference expressions that are present in the
expression, which is very useful extra context for expressions that get
evaluated multiple times, such as:
- Any expression in a block with "count" or "for_each" set
- The sub-expressions within a "for" expression
2018-10-18 17:12:01 -07:00
James Bardin e08a388d3c check IsKnown on values that may panic 2018-10-18 19:21:32 -04:00
Sander van Harmelen 48ef7ecfa6 Updates after running `make fmt` with Go v1.11.1 2018-10-17 14:11:08 -07:00
James Bardin fe9ed37dfc minor fixes for command Apply tests 2018-10-17 09:42:15 -04:00
Martin Atkins 541952bb8f Revert some work that happened since v0.12-dev branched
This work was done against APIs that were already changed in the branch
before work began, and so it doesn't apply to the v0.12 development work.

To allow v0.12 to merge down to master, we'll revert this work out for now
and then re-introduce equivalent functionality in later commits that works
against the new APIs.
2018-10-16 19:48:28 -07:00
Martin Atkins 8e51363f04 command: Don't allow -var and -var-file when applying saved plan
This reinstates an old behavior that was lost in the reorganization of how
we deal with the -var and -var-file options.

This fix is verified by TestApply_planVars now passing.
2018-10-16 19:14:11 -07:00
Martin Atkins 7abf81d8da command: Restore support for terraform.tfvars.json
In the new implementation of collecting variables I initially forgot the
JSON variant of terraform.tfvars.

This fix is verified by TestApply_varFileDefaultJSON now passing.
2018-10-16 19:14:11 -07:00
Martin Atkins 66f96cf842 command: Un-stub and reimplement "terraform state rm"
This was previously targeting the old state manager and state types, so it
needed some considerable rework to get it working again with the new state
types.

Since our new resource address syntax lacks the weird extra .deposed
special case we had before, we instead interpret addresses as
whole-instance addresses here and remove the deposed objects along with
the current one (if present), since this is more likely to match with
user expectations because we don't consider deposed objects to be
independently addressable in any other situation.

With that said, to be more explicit about what is going on we do now have
a -dry-run mode and maintain separate counts of current and deposed
instances so that we can expose that in the UI where relevant.
2018-10-16 19:14:11 -07:00
Martin Atkins 40eda180d6 command: go fmt in import_test.go and refresh_test.go 2018-10-16 19:14:11 -07:00
Martin Atkins 8b466811d2 command: Fix a few tests from meta_backend_test.go 2018-10-16 19:14:11 -07:00
Martin Atkins 86b7963bba command: Fix tests for "terraform providers" 2018-10-16 19:14:11 -07:00
Martin Atkins 5b61cc919b command: Fix "terraform import" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 7b77e20bdc command: Fix several "terraform init" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 89d0944e5b command: Fix even more of the "terraform refresh" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 5d1d6a95f9 command: "go fmt" of plan_test.go 2018-10-16 19:14:11 -07:00
Martin Atkins e54848b86f command: Fix many (but not all) "terraform refresh" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 73abb6e8f4 command: Re-enable showing outputs after successful "apply"
We temporarily disabled this because it needed some further work to update
it for the new state models, which has now been done.

We no longer need the configuration objects for the outputs because the
state itself contains all of the information needed for displaying these.
2018-10-16 19:14:11 -07:00
Martin Atkins 98bbd560b5 command: Fix most (but not all) "terraform plan" tests 2018-10-16 19:14:11 -07:00
Martin Atkins ca314afc0d command: Make suitable provider schemas available for all "plan" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 73318a436b command: go fmt 2018-10-16 19:14:11 -07:00
Martin Atkins 53bb3f57e6 command: Remove tests related to legacy remote state migrations, etc
We no longer support legacy remote state, so the behaviors these tests
were covering are no longer present.
2018-10-16 19:14:11 -07:00
Martin Atkins 275a44f552 command: Reinstate object ids in the UIHook progress logs
We used to treat the "id" attribute of a resource as special and elevate
it into its own struct field "ID" in the state, but the new state format
and provider protocol treats it just as any other attribute.

However, it's still useful to show the value of a single identifying
attribute when there isn't room in the UI for showing all of the
attributes, and so here we take a new strategy of considering "id" along
with some other conventional names as special only in the UI layer.

This new heuristic approach can be adjusted over time as new provider
patterns emerge, but for now it covers some common conventions we've seen
in real providers.

With that said, since all existing providers made for Terraform versions
prior to v0.12 were forced to set "id", we won't see any use of other
attributes here until providers are updated to remove the placeholder
ids they were generating in cases where an id was not actually relevant
but was forced by the old protocol. At that point the UX should be
improved by showing a more relevant attribute instead.

We now also allow for the possibility of no id at all, since that is valid
for resources that exist only within the Terraform state, like the ones
from the "random" and "tls" providers.
2018-10-16 19:14:11 -07:00
Martin Atkins 43d5206d82 command: Stub the "terraform state mv" tests to remind to fix
This command isn't yet updated for the new state types, but since we were
not returning a non-successful error status here the tests were just
failing in a weird way instead. Now we'll fail with a message that makes
it clear there is still work to do in the real implementation here.
2018-10-16 19:14:11 -07:00
Martin Atkins 1d64ce8ec0 command: Fix "terraform validate" tests
We now need to give a provider schema that the test fixtures conform to,
so we can validate against it.
2018-10-16 19:14:11 -07:00
Martin Atkins 025540789c command: Restore the "terraform output" functionality
We previously stubbed most of this out because it hadn't yet been updated
to support the new state types, etc.

This restores all of the previous behavior as covered by the tests.

We intentionally remove one behavior that was not covered by the tests:
we used to allow retrieval of outputs from non-root modules using the
-module option, but since we no longer persist non-root outputs in the
state we can no longer support this without a full expression evaluation
walk, and that'd be overkill for this otherwise-simple command. Descendant
module outputs are not part of the public interface of a configuration
anyway, so accessing them from outside in this way is an anti-pattern.

(For debugging scenarios it is still possible to access these from
"terraform console", which _does_ do a full evaluation graph walk to
prepare its evaluation scope.)
2018-10-16 19:14:11 -07:00
Martin Atkins feff10dcb5 command: Fix TestGraph_plan
The plan file writer requires a backend config to be present, but we don't
really need one for the sake of _this_ test, since we don't activate the
backend to render a plan graph, and so we just write in a placeholder.
2018-10-16 19:14:11 -07:00
Martin Atkins c0baedac84 command: Fix tests for "terraform fmt" command
Some underlying assumptions have shifted here, so although the behavior
is still broadly the same we need to accommodate some different details.
2018-10-16 19:14:11 -07:00
Martin Atkins 74582447bb command: Make input variable values available to "terraform console"
This fixes TestConsole_tfvars.
2018-10-16 19:14:11 -07:00
Martin Atkins 741d334ee4 command: Even more fixes for "apply" command tests 2018-10-16 19:14:11 -07:00
Martin Atkins 34a29315f7 command: More fixes (but still not all) for "apply" command tests 2018-10-16 19:14:11 -07:00
Martin Atkins de3944b9bf command: collect root module variable values for apply/plan/refresh
This connects a missing link left by earlier refactoring: the command
package is responsible for gathering up variable values provided by the
user and passing them through to the backend to use in operations.
2018-10-16 19:14:11 -07:00
Martin Atkins fc2614c939 command: Fix some tests for the "apply" command 2018-10-16 19:14:11 -07:00
Martin Atkins 34c4fd316d command: Update backend-unchanged fixture for new expectations
Our serialization of the backend configuration has changed slightly for
Terraform 0.12 due to reimplementing it in terms of the HCL2 types, so
the base case that should be unchanged during the test needs to be
changed.
2018-10-16 19:14:11 -07:00
Martin Atkins 34ebde0b95 command/format: be resilient to incomplete schema when formatting state
In all real cases the schemas should be populated here, but we don't want
to panic in UI rendering code if there's a bug here.

This can also be tripped up by tests with incomplete mocks. It's
unfortunate that this can therefore mask some problems in tests, but tests
can protect against it by asserting on specific output text rather than
just assuming that a zero exit status is a pass.
2018-10-16 19:14:11 -07:00
Kristin Laemmert 3cf1b001c2 command/format: revert indentation change for consistency's sake 2018-10-16 19:14:11 -07:00
Kristin Laemmert db26324b3c command/format: fix indentation
also run go mod tidy
2018-10-16 19:14:11 -07:00
Martin Atkins d63c2fdd8d command: Show snippet of invalid resource addresses in import
If we fail to parse the resource address given to "terraform import" then
it's helpful to produce a "source code" snippet of what the user provided
so they might see more precisely which part of the address was invalid.
2018-10-16 19:14:11 -07:00
Martin Atkins 27d086d8bd command: Fix TestUntaint 2018-10-16 19:14:11 -07:00
Martin Atkins 1bc10180cb command: Fix all of the "taint" command tests
Most of this is just updates to allow for the fact that we now always save
the provider address as part of resource state, whereas before it was only
saved conditionally.

This also updates TestTaint_module for the intentional change that it now
expects a child module to be specified using normal resource address
syntax, rather than as a separate -module option.
2018-10-16 19:14:11 -07:00
Martin Atkins 8321c3614f command: Fix TestUntaint_module
The untaint command now uses normal resource address syntax to specify the
module path, rather than passing it in as a separate argument.
2018-10-16 19:14:11 -07:00
Martin Atkins 3cee60ee3a command: Fix various "untaint" command tests
Resource instance state now always includes the provider address fo each
resource, so our expected state output strings needed to be adjusted.
2018-10-16 19:14:11 -07:00
Martin Atkins 55b9c7eacd command: Actually include the error when backend config decoding fails 2018-10-16 19:14:11 -07:00
Kristin Laemmert a4c3022cf8 command/show: config files are assumed to be in cwd 2018-10-16 19:14:11 -07:00
Kristin Laemmert d08fe7a91f cli: format/state_test.go
Added a very simple test with state and schema.
TODO: if tests are added we should test using golden files (and example
state files, instead of strings). This seemed unnecessary with the
simple test cases.
2018-10-16 19:14:11 -07:00
Kristin Laemmert 14c28b8de4 cli: format/state refactor to use blockBodyDiffPrinter
Use functions from format/diff to print values
2018-10-16 19:14:11 -07:00
Kristin Laemmert 8063f69e5c cli: format/state refactor for new state format
format/state now requires provider schemas to properly format output
state. command/show has been modified to pass a context to format.State.
2018-10-16 19:14:11 -07:00
Martin Atkins e8240087fe command: Populate backend configuration in plan files
In previous work we didn't quite connect these dots. The connection here
is sub-awesome since the existing interfaces here had some unfortunate
assumptions that we'd like to move away from (like the idea of a "nil
backend" implying the local backend) but we're accepting this for now to
avoid another big round of refactoring.

The main implication of this is that we will now always include a backend
configuration in the plan, though it might just be a placeholder config
for the local backend in the remaining cases where that's still implicitly
set. Later we will change this so that there is no implicit local backend
at all (terraform init is always required, _it_ will deal with setting
implicitly setting the local backend when appropriate), which will allow
us to rework this to be more straightforward and less "spooky".
2018-10-16 19:14:11 -07:00
Martin Atkins cbc548eb36 command: Do CLI init of backend loaded from plan
If we don't do this, we can't produce any output when applying a saved
plan file.

Here we also introduce a check to the local backend's ReportResult
function so that it won't panic if CLI init is skipped, although that
will no longer happen in the apply-from-file case due to the change
described in the previous paragraph.
2018-10-16 19:14:11 -07:00
James Bardin 52c0032aed update provisioners for multiple processes
The "internal" provisioners are still run in a separate process, and
need to be updated to restart on each walk.
2018-10-16 19:14:11 -07:00
James Bardin 795161f615 update to start a new process for each plugin
Modify the plugin factories to create a new plugin process for each
individual plugin.
2018-10-16 19:14:11 -07:00
Martin Atkins 412439ae01 command: Un-stub tests that were previously hanging on the inmem backend
The underlying bug has now been fixed,
in b3b62429a7bbeb852f9b26299732b265d9d3299f .
2018-10-16 19:14:11 -07:00
Martin Atkins 058a1d81b0 command/format: Remove tests for plan.go
plan.go as a whole will be removed before too long, so there's no point
in us updating all of these tests to use the new plan types.
2018-10-16 19:14:11 -07:00
Martin Atkins 3b8d46418f command/format: print correctly nested blocks with labels 2018-10-16 19:14:11 -07:00
Martin Atkins b0016e9cf6 command: Allow tests to run to completion without panics or hangs
There are still 160 test failures as of this commit, but at least the test
program can run to completion and list out all the failures.
2018-10-16 19:14:11 -07:00
Martin Atkins d446d32d96 command: Fix TestApply for new mock provider interface
It must now provide a basic implementation of plan and apply for its
mock provider, which in this case can just pass through the proposed value
generated by core because there are no computed attributes in this schema.
2018-10-16 19:14:11 -07:00
Martin Atkins 5021e0e098 command: Fix TestApply_destroyTargeted for new provider and state types 2018-10-16 19:14:11 -07:00
Martin Atkins db3ea65e8b command: Fix TestApply_destroy for new provider and state types 2018-10-16 19:14:11 -07:00
Martin Atkins be7bce493e command: Update plugins_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins dc0b5292a6 command: Update refresh_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins b195c712a3 command: Update plan_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins 54c1616d73 command: Update import_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins 65341b0973 command: Update apply_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins d9051be66d command: Use new provider and provisioner types in test helpers
This moves our build errors from the test helpers out into the tests
themselves. We'll update the tests in subsequent commits.
2018-10-16 19:14:11 -07:00
Martin Atkins a43b7df282 core: Handle forced-create_before_destroy during the plan walk
Previously we used a single plan action "Replace" to represent both the
destroy-before-create and the create-before-destroy variants of replacing.
However, this forces the apply graph builder to jump through a lot of
hoops to figure out which nodes need it forced on and rebuild parts of
the graph to represent that.

If we instead decide between these two cases at plan time, the actual
determination of it is more straightforward because each resource is
represented by only one node in the plan graph, and then we can ensure
we put the right nodes in the graph during DiffTransformer and thus avoid
the logic for dealing with deposed instances being spread across various
different transformers and node types.

As a nice side-effect, this also allows us to show the difference between
destroy-then-create and create-then-destroy in the rendered diff in the
CLI, although this change doesn't fully implement that yet.
2018-10-16 19:14:11 -07:00
Martin Atkins b428746cfd command: UiHook to report when it gets incorrect values
For PreApply hook purposes we only actually use the Delete, Create, and
Update actions, because other actions are handled in different ways than
a direct call to ApplyResourceChange.

However, if there's a bug in core that causes it to pass a different
action, it's better for us to mark it as being explicitly unknown in the
UI rather than simply defaulting to "Modifying...", which can thus obscure
the problem and make for a confusing result.
2018-10-16 19:14:11 -07:00
Martin Atkins a37d5268eb command/format: Render unified diff for list and tuple value changes 2018-10-16 19:14:11 -07:00
Martin Atkins 30a46ed8ed command/format: Further refinement on the heuristic JSON-based diff
We'll now show an "update" symbol prior to the argument to this synthetic
jsonencode(...) call, for consistency with how we show nested values in
other cases and to attach a verb to any "# forces replacement".

We'll also show a special form in the case where the value seems to differ
only in whitespace, so users can understand what's going on in that
hopefully-rare situation, particularly if those whitespace-only changes
end up forcing us to replace a remote object.
2018-10-16 19:14:11 -07:00
Martin Atkins 783205948c command/format: JSON value formatting heuristic not for primitive values
Since our own syntax for primitive values is similar to that of JSON, and
since we permit automatic conversions from number and bool to string, we
must do this special JSON value diff formatting only if the value is a
JSON array or object to avoid confusing results.
2018-10-16 19:14:11 -07:00
Martin Atkins 4406b06ef8 command/format: Don't include commas after elements in map diffs
Although commas are allowed in this context, it's not idiomatic to use
them.
2018-10-16 19:14:11 -07:00
Martin Atkins ce157c7f23 command/format: show deep diffs for old/new JSON strings
Because so far we've not supported dynamically-typed complex data
structures, several providers have used strings containing JSON to stand
in for these.

In order to get a readable diff in those cases, we'll recognize situations
where old and new are both JSON and present a diff of the effective value
of the JSON, using a faux call to the jsonencode(...) function to indicate
when we've done so.

This is a bit of a "cute" heuristic, but is important at least for now
until we can migrate away from that practice of passing large JSON strings
to providers and use dynamically-typed attributes instead.
2018-10-16 19:14:11 -07:00
Martin Atkins 9706a00b3a command/format: correct alignment for multi-line values in map diff 2018-10-16 19:14:11 -07:00
Martin Atkins 04d8c17be8 command/format: Nice rendering for changes to map values 2018-10-16 19:14:11 -07:00
Martin Atkins a2f4b8b058 command/format: Show resource diff with header "comment"
This extra comment line gives us a place to show the full resource address
(since the block header line only includes type and name) and also allows
us to explain in long form the meaning of the change icon on the following
line.
2018-10-16 19:14:11 -07:00
Martin Atkins 239a54ad6f command: initial structural diff rendering
This is a light adaptation of our earlier prototype of structural diff
rendering, as a starting point for what we'll actually ship. This is not
consistent with the latest mocks, so will need some additional work before
it is ready, but integrating this allows us to at least see the plan
contents while fixing up remaining issues elsewhere.
2018-10-16 19:14:11 -07:00
Martin Atkins a7342de274 core: Properly handle no-op changes in plan
Previously we just left these out of the plan altogether, but in the new
plan types we intentionally include change information for every resource
instance, even if no changes are actually planned, to allow alternative
plan file viewers to show what isn't changing as well as what is.
2018-10-16 19:14:11 -07:00
Martin Atkins 5731703de5 command/format: show planned changes in rendered plan
This codepath is going to be significantly changed before release to make
it support structural diff of the new data types, but this lets us lean on
the old renderer to produce partial output in the mean time while we
continue to work on getting things working end-to-end after the
considerable refactoring that's been going on.
2018-10-16 19:14:11 -07:00
Martin Atkins 44bc7519a6 terraform: More wiring in of new provider types
This doesn't actually work yet, but it builds and then panics in a pretty
satisfying way.
2018-10-16 19:12:54 -07:00
Martin Atkins a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Kristin Laemmert ce5e66e178 plugin/discovery provider installer: download providers from the registry
Terraform will query the public registry at
https://registry.terraform.io for providers, instead of
https://releases.hashicorp.com.
2018-10-16 18:56:50 -07:00
Martin Atkins 479c6b2466 move "configschema" from "config" to "configs"
The "config" package is no longer used and will be removed as part
of the 0.12 release cleanup. Since configschema is part of the
"new world" of configuration modelling, it makes more sense for
it to live as a subdirectory of the newer "configs" package.
2018-10-16 18:50:29 -07:00
Martin Atkins adb88eaa16 configupgrade: Analysis of input configuration
In order to properly migrate the contents of resource, data, provider and
provisioner blocks we will need the provider's schema in order to
understand what is expected, so we can resolve some ambiguities inherent
in the legacy HCL AST.

This includes an initial prototype of migrating the content of resource
blocks just to verify that the information is being gathered correctly.
As with the rest of the upgrade_native.go file, this will be reorganized
significantly once the basic end-to-end flow is established and we can
see how to organize this code better.
2018-10-16 18:50:29 -07:00
Martin Atkins eac8779870 command: validate must set values for root variables
Since the intent of the validate command is to check config validity
regardless of context (input variables, state, etc), we use unknown values
of the requested type here, which will then allow us to complete type
checking against the specified types of the variables without assuming
any particular values.
2018-10-16 18:50:29 -07:00
Martin Atkins ffe5f7c4e6 command: 0.12upgrade command
This is the frontend to the work-in-progress codepath for upgrading the
source code for a module written for Terraform v0.11 or earlier to use
the new syntax and idiom of v0.12.

The underlying upgrade code is not yet complete as of this commit, and
so the command is not yet very useful. We will continue to iterate on
the upgrade code in subsequent commits.
2018-10-16 18:50:29 -07:00
Martin Atkins 0742e756e5 tfdiags: Sort order for diagnostics
Because we gather together diagnostics from many different parts of the
codebase, the list often ends up being in a non-ideal order. Here we
define a partial ordering for diagnostics that should hopefully make them
easier to scan when many are present, by grouping together diagnostics
that are of the same severity and belong to the same file.

We use sort.Stable here because we have a partial order and so we need
to make sure that diagnostics that do not have a relative ordering will
remain in their original order.

This sorting is applied just in time before rendering the diagnostics
in command.Meta.showDiagnostics.
2018-10-16 18:50:29 -07:00
Martin Atkins 2ef56b3e05 Fix up some missed "go fmt"
Because of the size of some of these files, automatic format-on-save was
implicitly disabled in my editor, which I didn't notice before committing.
2018-10-16 18:49:20 -07:00
Martin Atkins dc7f793be9 command: terraform fmt to use new HCL formatter
This doesn't yet include test updates, since there are problems in core
currently blocking these tests from running. The tests will therefore be
updated in a subsequent commit.
2018-10-16 18:49:20 -07:00
Martin Atkins a270a18a4d command: Update tests for changes in "terraform" package
This is not exhaustive, but it gets the tests in this package compiling
again and gets most of them working.
2018-10-16 18:49:20 -07:00
Martin Atkins fc7871c5ee command/import: default provider config in same module
Previously we were defaulting the provider configuration selection to a
provider in the root module inferred from the resource type name.

This is close, but not quite right: we need to _start_ with a provider
configuration in the same module as we're importing into, and then our
provider resolution steps during import graph construction will use that
as a starting point for a walk up the tree to find the nearest matching
configuration (which might eventually still be in the root, but not
necessarily).
2018-10-16 18:48:28 -07:00
Martin Atkins f085af4ba6 command: update "terraform console" for HCL2
This now uses the HCL2 parser and evaluator APIs and evaluates in terms
of a new-style *lang.Scope, rather than the old terraform.Interpolator
type that is no longer functional.

The Context.Eval method used here behaves differently than the
Context.Interpolater method used previously: it performs a graph walk
to populate transient values such as input variables, local values, and
output values, and produces its scope in terms of the result of that
graph walk. Because of this, it is a lot more robust than the prior method
when asked to resolve references other than those that are persisted
in the state.
2018-10-16 18:46:46 -07:00
Martin Atkins 9ca13d5c1d command/validate: make sure diagnostics is always present and an array
Previously an empty diagnostics would appear as "null" in the JSON output,
since that is how encoding/json serializes a nil slice. It's more
convenient for users of dynamic languages to keep the type consistent
in all cases, since they can then just iterate the list without needing a
special case for when it is null.
2018-10-16 18:46:46 -07:00
Martin Atkins c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins 4ed06a9227 terraform: HCL2-flavored module dependency resolver
For the moment this is just a lightly-adapted copy of
ModuleTreeDependencies named ConfigTreeDependencies, with the goal that
the two can live concurrently for the moment while not all callers are yet
updated and then we can drop ModuleTreeDependencies and its helper
functions altogether in a later commit.

This can then be used to make "terraform init" and "terraform providers"
work properly with the HCL2-powered configuration loader.
2018-10-16 18:44:26 -07:00
Martin Atkins ebafa51723 command: Various updates for the new backend package API
This is a rather-messy, complex change to get the "command" package
building again against the new backend API that was updated for
the new configuration loader.

A lot of this is mechanical rewriting to the new API, but
meta_config.go and meta_backend.go in particular saw some major
changes to interface with the new loader APIs and to deal with
the change in order of steps in the backend API.
2018-10-16 18:44:26 -07:00
Martin Atkins 5782357c28 backend: Update interface and implementations for new config loader
The new config loader requires some steps to happen in a different
order, particularly in regard to knowing the schema in order to
decode the configuration.

Here we lean directly on the configschema package, rather than
on helper/schema.Backend as before, because it's generally
sufficient for our needs here and this prepares us for the
helper/schema package later moving out into its own repository
to seed a "plugin SDK".
2018-10-16 18:39:12 -07:00
Martin Atkins 618883596a command: remove "terraform push"
The remote API this talks to will be going away very soon, before our next
major release, and so we'll remove the command altogether in that release.

This also removes the "encodeHCL" function, which was used only for
adding a .tfvars-formatted file to the uploaded archive.
2018-10-16 18:24:47 -07:00
Martin Atkins 9a004e3f9d command: "terraform validate" JSON support
In the long run we'd like to offer machine-readable output for more
commands, but for now we'll just start with a tactical feature in
"terraform validate" since this is useful for automated testing scenarios,
editor integrations, etc, and doesn't include any representations of types
that are expected to have breaking changes in the near future.
2018-10-16 18:20:32 -07:00
Martin Atkins bfd9392eb8 command: beginnings of new config loader in "terraform validate"
As part of some light reorganization of our commands, this new
implementation no longer does validation of variables and will thus avoid
the need to spin up a fully-valid context. Instead, its focus is on
validating the configuration itself, regardless of any variables, state,
etc.

This change anticipates us later adding a -validate-only flag to
"terraform plan" which will then take over the related use-case of
checking if a particular execution of Terraform is valid, _including_ the
state, variables, etc.

Although leaving variables out of validate feels pretty arbitrary today
while all of the variable sources are local anyway, we have plans to
allow per-workspace variables to be stored in the backend in future and
at that point it will no longer be possible to fully validate variables
without accessing the backend. The "terraform plan" command explicitly
requires access to the backend, while "terraform validate" is now
explicitly for local-only validation of a single module.

In a future commit this will be extended to do basic type checking of
the configuration based on provider schemas, etc.
2018-10-16 18:20:32 -07:00
Martin Atkins fd5b7b42b8 command: new Meta methods for accessing the new config loader
We need to share a single config loader across all callers because that
allows us to maintain the source code cache we'll use for snippets in
error messages.

Nothing calls this yet. Callers will be gradually updated away from Module
and Config in subsequent commits.
2018-10-16 18:20:32 -07:00
Martin Atkins bd10b84a8e command/format: include source snippets in diagnostics
If we get a diagnostic message that references a source range, and if the
source code for the referenced file is available, we'll show a snippet of
the source code with the source range highlighted.

At the moment we have no cache of source code, so in practice this
codepath can never be visited. Callers to format.Diagnostic will be
gradually updated in subsequent commits.
2018-10-16 18:20:32 -07:00
Sander van Harmelen b1fdbd7db8 Allow enhanced backends to pass custom exit codes
In some cases this is needed to keep the UX clean and to make sure any remote exit codes are passed through to the local process.

The most obvious example for this is when using the "remote" backend. This backend runs Terraform remotely and stream the output back to the local terminal.

When an error occurs during the remote execution, all the needed error information will already be in the streamed output. So if we then return an error ourselves, users will get the same errors twice.

By allowing the backend to specify the correct exit code, the UX remains the same while preserving the correct exit codes.
2018-10-05 20:44:12 +02:00
Sander van Harmelen 67db9da000 Add checks for all flags we currently don’t support
For Plan only:
-module-depth=n

For Plan & Apply
-parallelism=m
-refresh=false
-var “foo=bar” and -var-file=foo
2018-10-05 20:16:34 +02:00
Kevin Carmody 7ac9a576bb
Update wrong usage workspace command description
Small update to the description to include available commands, rather than similar looking but wrong ones.
2018-10-03 12:00:27 +01:00
Sander van Harmelen 36bd5d6480 Make sure the state is locked before it is used
Both the `taint` and `untaint` commands had a small logical error that caused them you first get the state and then lock it.
2018-09-18 20:32:04 +02:00
Sean Chittenden d749420a25
Fix drift caused from gofmt when running make dev and go 1.11.
A fresh checkout of `origin/master` does not build atm using the `dev`
target because `master` has not been formatted using `gofmt` from Go
1.11 (tis has been the case for a while if you've been running devel).

None of the drift in question is especially new but now that Go 1.11
has been released and gofmt's formatting guidelines have been updated,
it would be *really* nice if the code in `master` reflected the current
tooling in order to avoid having to fight this drift locally.

* 8mo: https://github.com/hashicorp/terraform/blame/master/backend/remote-state/s3/backend_test.go#L260-L261
* 6mo: https://github.com/hashicorp/terraform/blame/master/builtin/provisioners/chef/linux_provisioner_test.go#L124
* 1yr: 7cfeffe36b/command/init.go (L75-L76)
* 12d: 7cfeffe36b/command/meta_backend_test.go (L1437)
* 2yr: 7cfeffe36b/helper/schema/resource_timeout_test.go (L26)
* 4yr: 7cfeffe36b/helper/schema/schema_test.go (L2059)
* 1yr: 7cfeffe36b/plugin/discovery/get_test.go (L151)
2018-09-09 10:18:08 -07:00
Oleg Butuzov 2ac15fdae4 command: Fix inconsistent spacing in "terraform push -help"
Fixes #18775
2018-09-04 18:47:46 -07:00
Sander van Harmelen ce2869dced
Merge pull request #18760 from hashicorp/f-remote-backend
backend/migrations: migrate the default state
2018-08-30 11:16:25 +02:00
Sander van Harmelen f410a5bb26 backend/migrations: migrate the default state
Certain backends (currently only the `remote` backend) do not support using both the default and named workspaces at the same time.

To make the migration easier for users that currently use both types of workspaces, this commit adds logic to ask the user for a new workspace name during the migration process.
2018-08-29 21:37:39 +02:00
Sander van Harmelen 1696ade924 backend/migrations: check all workspaces
This commit fixes a bug that (in the case of the `local` backend) would only check if the selected workspace had a state when deciding to preform a migration.

When the selected workspace didn’t have a state (but other existing workspace(s) did), the migration would not be preformed and the other workspaces would be ignored.
2018-08-29 19:45:35 +02:00
Sander van Harmelen 7fb2d1b8de Implement the Enterprise enhanced remote backend 2018-08-03 22:22:55 +02:00
Sander van Harmelen 179b32d426 Add a `CredentialsForHost` method to disco.Disco
By adding this method you now only have to pass a `*disco.Disco` object around in order to do discovery and use any configured credentials for the discovered hosts.

Of course you can also still pass around both a `*disco.Disco` and a `auth.CredentialsSource` object if there is a need or a reason for that!
2018-08-03 11:29:11 +02:00
Sander van Harmelen 495d1ea350 Use New() instead of `once.Do(b.init)` 2018-08-03 11:29:11 +02:00
Sander van Harmelen 97d1c46602 Update the backend import names
It’s a purely cosmetic change, but I find it easier to read them like this.
2018-08-03 11:29:11 +02:00
Aurelie Vache 7a29a03e21 command: help text for "workspace" to include "list" verb
One of the subcommands lists out the existing workspaces.
2018-07-09 17:45:13 -07:00
Kristin Laemmert 33877bd579 cli: update tests 2018-06-26 11:57:52 -07:00
Lyle Franklin 26b907387d cli: Remove error on empty outputs when `-json` is set (#11721)
- Fixes #11696
- This changes makes `terraform output -json` return '{}' instead of
  throwing an error about "no outputs defined"
- If `-json` is not set, the user will receive an error as before
  - This UX helps new users to understand how outputs are used
- Allows for easier automation of TF CLI as an empty set of outputs is
  usually acceptable, but any other error from `output` would be
  re-raised to the user.
2018-06-20 08:58:07 -07:00
Kristin Laemmert b8a1a77f6b
command: fix spelling in error message - fixes #18065 (#18241) 2018-06-13 06:57:32 -07:00
James Bardin 545e77e184
Merge pull request #17711 from hashicorp/jbardin/cleanup-temps
cleanup temp files
2018-03-28 14:06:21 -04:00
James Bardin e980156451 cleanup temp files from command tests
Rather than try to modify all the hundreds of calls to the temp helper
functions, and cleanup the temp files at every call site, have all tests
work within a single temp directory that is removed at the end of
TestMain.
2018-03-28 13:08:38 -04:00
Masayuki Morita 1e2bb07504 command: Restore `auto-approve` flag in the output of `apply -help` 2018-03-26 10:21:47 -07:00
James Bardin 90a75422fb unlock state in console, import, graph, and push
The state locking improvements for the regular command had the side
effect of locking the state in the console, import, graph and push
commands. Those commands had been updated to get a state via the
Backend.Context method, which locks the state whenever possible, and now
need to call Unlock directly.

Add Unlock calls to all commands that call Context directly.
2018-03-21 12:13:40 -04:00
Martin Atkins 6aefa5835c Merge #17218: Add -auto-approve to "terraform destroy" for consistency 2018-03-08 17:42:15 -08:00
James Bardin dc8036636a
Merge pull request #17422 from hashicorp/jbardin/state-locking
Add StateLocker interface to backend operations
2018-02-27 11:38:55 -05:00
James Bardin 2b97585d46 improve clistate.Locker docs 2018-02-27 10:49:06 -05:00
Uriel Corfa 07a20365de cli: "terraform state list" -id argument
A new -id option to "terraform state list" which constraints the output of the command to resources whose id is equal to the flag value.
2018-02-26 10:54:48 -08:00
James Bardin bdd475e149 use the new clistate.Locker in commands
Use the new StateLocker field to provide a wrapper for locking the state
during terraform.Context creation in the commands that directly
manipulate the state.
2018-02-23 16:48:15 -05:00
James Bardin e9a76808df create clistate.Locker interface
Simplify the use of clistate.Lock by creating a clistate.Locker
instance, which stores the context of locking a state, to allow unlock
to be called without knowledge of how the state was locked.

This alows the backend code to bring the needed UI methods to the point
where the state is locked, and still unlock the state from an outer
scope.

Provide a NoopLocker as well, so that callers can always call Unlock
without verifying the status of the lock.

Add the StateLocker field to the backend.Operation, so that the state
lock can be carried between the different function scopes of the backend
code. This will allow the backend context to lock the state before it's
read, while allowing the different operations to unlock the state when
they complete.
2018-02-23 16:48:15 -05:00
James Bardin 12bda53558 Revert "create clistate.Locker interface"
This reverts commit e88bd74bb7.
2018-02-23 11:31:13 -05:00
James Bardin 3de0d24554 Revert "use the new clistate.Locker in commands"
This reverts commit 400f6ca4c5.
2018-02-23 11:31:06 -05:00
James Bardin 400f6ca4c5 use the new clistate.Locker in commands
Use the new StateLocker field to provide a wrapper for locking the state
during terraform.Context creation in the commands that directly
manipulate the state.
2018-02-23 11:28:47 -05:00
James Bardin e88bd74bb7 create clistate.Locker interface
Simplify the use of clistate.Lock by creating a clistate.Locker
instance, which stores the context of locking a state, to allow unlock
to be called without knowledge of how the state was locked.

This alows the backend code to bring the needed UI methods to the point
where the state is locked, and still unlock the state from an outer
scope.
2018-02-23 11:21:21 -05:00
James Bardin a37acb1837 gofmt with go1.10 2018-02-21 10:22:08 -05:00
James Bardin ecd9ef0f77 ignore error in plan shutdown test
The error was being silently dropped before.

There is an interpolation error, because the plan is canceled before
some of the resources can be evaluated. There might be a better way to
handle this in the walk cancellation, but the behavior has not changed.

Make the plan and apply shutdown match implementation-wise
2018-02-12 11:56:54 -05:00
James Bardin 67a6152091 move backend operation cancellation into meta
Create a single command method for running and operation with
cancellation.
2018-02-12 11:56:54 -05:00
James Bardin 7cba68326a always wait for a RunningOperation to return
If the user wishes to interrupt the running operation, only the first
interrupt was communicated to the operation by canceling the provided
context. A second interrupt would start the shutdown process, but not
communicate this to the running operation. This order of event could
cause partial writes of state.

What would happen is that once the command returns, the plugin system
would stop the provider processes. Once the provider processes dies, all
pending Eval operations would return return with an error, and quickly
cause the operation to complete. Since the backend code didn't know that
the process was shutting down imminently, it would continue by
attempting to write out the last known state. Under the right
conditions, the process would exit part way through the writing of the
state file.

Add Stop and Cancel CancelFuncs to the RunningOperation, to allow it to
easily differentiate between the two signals. The backend will then be
able to detect a shutdown and abort more gracefully.

In order to ensure that the backend is not in the process of writing the
state out, the command will always attempt to wait for the process to
complete after cancellation.
2018-02-12 11:56:03 -05:00
Laura Martin 6e1e614a56 Change -force to -auto-approve when destroying
Since an early version of Terraform, the `destroy` command has always
had the `-force` flag to allow an auto approval of the interactive
prompt. 0.11 introduced `-auto-approve` as default to `false` when using
the `apply` command.

The `-auto-approve` flag was introduced to reduce ambiguity of it's
function, but the `-force` flag was never updated for a destroy.

People often use wrappers when automating commands in Terraform, and the
inconsistency between `apply` and `destroy` means that additional logic
must be added to the wrappers to do similar functions. Both commands are
more or less able to run with similar syntax, and also heavily share
their code.

This commit updates the command in `destroy` to use the `-auto-approve` flag
making working with the Terraform CLI a more consistent experience.

We leave in `-force` in `destroy` for the time-being and flag it as
deprecated to ensure a safe switchover period.
2018-02-01 00:14:42 +00:00
James Bardin 6f41a80b3a slow down the plan shutdown tests for slow hosts
The plan shutdown test often fail on slow CI hosts, becase the plan
completes befor the main thread can cancel it. Since attempting to make
the MockProvider concurrent proved too invasive for now, just slow the
test down a bit to help ensure Stop gets called.
2018-01-31 17:09:48 -05:00
Masayuki Morita 338d5cad70 Remove duplicated help messages from `workspace --help`
Fixes #17165
2018-01-23 23:38:42 +09:00
Masayuki Morita f440dba137 Standardize on log level "WARN" rather than "WARNING" 2018-01-16 18:05:26 -08:00
Martin Atkins a6008b8a48
v0.11.2 2018-01-09 23:13:33 +00:00
James Bardin 61c2be3e95
Merge pull request #16969 from hashicorp/jbardin/reset-plugin-dir
allow init to reset -plugin-dir
2018-01-08 16:51:47 -05:00
James Bardin 7d5f7cb22f
Merge pull request #16961 from hashicorp/jbardin/mock-provider-race
minor race issue in mockResourceProvider
2018-01-08 16:47:58 -05:00
James Bardin 504ea578ee
Merge pull request #16920 from hashicorp/jbardin/init-future-state
check state version during init
2018-01-05 16:54:41 -05:00
James Bardin dec2da1dda
Merge pull request #17044 from hashicorp/jbardin/plugin-dir-internal
-plugin-dir short-circuits discovery of internal plugins
2018-01-05 16:54:28 -05:00
James Bardin 6367572455
Merge pull request #16882 from Techcadia/f-provider-logging
[Add] Pathing and Override to DEBUG Command
2018-01-05 12:25:24 -05:00
James Bardin fe3aff91f3 missingPlugins should not return internal plugins
Filter the internal plugins out beforehand, so we don't attempt to
locate them at all during init.
2018-01-05 11:52:11 -05:00
James Bardin 5a975d9997 add test for internal plugins with -plugin-dir
-plugin-dir was short-circuiting the discovery for internal plugins
2018-01-05 11:51:09 -05:00
James Bardin 5a9e0cf763 create a new InternalProviders test
Remove the legacy InternalProviders global.

The terraform provider is the only one run internally.
2018-01-05 10:59:30 -05:00
James Bardin ba84faf4e1 remove the plugin path only for an empty string
To avoid breaking automation where plugin-path was assumed to be set
permanently, only remove the plugin-path record if it was explicitly set
to and empty string.
2018-01-04 16:49:45 -05:00
James Bardin f62b71710a
Merge pull request #16939 from hashicorp/jbardin/migrate-confirm
remove extra backend migration prompts
2018-01-04 16:27:25 -05:00
James Bardin f45205feb7 re-word state migration prompts
The existing prompts were worded as if backend configurations were
named, but they can only really be referenced by their type. Change the
wording to reference them as type "X backend". When migrating state,
refer to the backends as the "previously configured" and "newly
configured", since they will often have the same type.
2018-01-04 16:22:54 -05:00
James Bardin 4b49a323c3 go fmt
slight change to go fmt coming in 0.10
2017-12-26 13:26:38 -05:00
James Bardin 79e985366f allow init to reset -plugin-dir
Remove the recorded -plugin-dir during init if the flag is not provided.
2017-12-21 11:21:07 -05:00
James Bardin d76482cd89 don't try to interrupt diff in shutdown test
Rather than relying on interrupting Diff, just make sure Stop was called
on the provider. The DiffFn is protected by a mutex in the mock
provider, which means that the tests can't rely on concurent calls to
diff working.
2017-12-20 16:23:55 -05:00
James Bardin e63a3474d5 kill the flag error writer after 2 seconds
There's no point in trying to track these, they're lost after each test.
Kill them after a short delay so we don't have goroutines from every single
command test to wade through if we have a stack dump.
2017-12-20 15:52:43 -05:00
James Bardin 885e4cde81 don't loop indefinitely in confirm method
Only check for input twice in the meta.confirm method. This prevents an
errant newline from aborting the run while allowing Terraform to exit if
there is no input available. We don't just check for a tty, since we
still rely on being able to pipe input in for testing.

Remove the redundant confirmation loops in the migration code, and only
use the confirm method.
2017-12-18 18:39:21 -05:00
James Bardin 7d2da9865e inputFalse test should attempt migration and check error
Make sure the init inputFalse test actually errors from missing input,
since skipping input will still fail later during provider
initialization. We need to make sure there are two different states that
aren't a noop for migration, and reset the command struct for each run.

Also verify that we don't go into an infinite loop if there is no input.
2017-12-18 18:39:21 -05:00
James Bardin 909dff36a8 remove extra "yes" confirmation in tests
There were two tests that had the duplicate confirmations hard-coded in
the input stream.
2017-12-18 11:42:33 -05:00
James Bardin 7c93b2e5e6 remove duplicate backend migration prompts
The duplicate prompts can be confusing when the user confirms that a
migration should happen and we immediately prompt a second time for the
same thing with slightly different wording. The extra hand-holding that
this provides for legacy remote states is less critical now, since it's
been 2 major release cycles since those were removed.
2017-12-18 11:42:33 -05:00
James Bardin ea4cb6a20e check state version during init
The init command needs to parse the state to resolve providers, but
changes to the state format can cause that to fail with difficult to
understand errors. Check the terraform version during init and provide
the same error that would be returned by plan or apply.
2017-12-15 11:17:59 -05:00
Martin Atkins f1079257ac command: show a special error message when importing in an empty dir
If users run "terraform import" in a directory with no Terraform
configuration files, it's likely that they've made a mistake either by
being in the wrong directory or forgetting to use the -config option
on the command line.

To help users find their mistake in this case, we'll now produce a
specialized error message for this situation:

    Error: No Terraform configuration files

    The directory /home/user/example does not contain any Terraform
    configuration files (.tf or .tf.json). To specify a different
    configuration directory, use the -config="..." command line option.

While here, this also converts some of the other existing messages to
diagnostics so that we can show any configuration warnings along with
the error message, and move towards the new standard error presentation.
2017-12-11 16:08:33 -08:00
Techcadia e681907c50 [Add] Pathing and Override Output to DEBUG Command 2017-12-08 13:12:23 -08:00
Nolan Davidson 653db95df7 Initial implementation of a habitat provisioner
First pass at loading the config data using the TF schema.

Signed-off-by: Nolan Davidson <ndavidson@chef.io>
2017-12-07 16:29:30 -08:00
Martin Atkins 9a5c865040 command: validate config as part of loading it
Previously we required callers to separately call .Validate on the root
module to determine if there were any value errors, but we did that
inconsistently and would thus see crashes in some cases where later code
would try to use invalid configuration as if it were valid.

Now we run .Validate automatically after config loading, returning the
resulting diagnostics. Since we return a diagnostics here, it's possible
to return both warnings and errors.

We return the loaded module even if it's invalid, so callers are free to
ignore returned errors and try to work with the config anyway, though they
will need to be defensive against invalid configuration themselves in
that case.

As a result of this, all of the commands that load configuration now need
to use diagnostic printing to signal errors. For the moment this just
allows us to return potentially-multiple config errors/warnings in full
fidelity, but also sets us up for later when more subsystems are able
to produce rich diagnostics so we can show them all together.

Finally, this commit also removes some stale, commented-out code for the
"legacy" (pre-0.8) graph implementation, which has not been available
for some time.
2017-12-07 14:28:43 -08:00
James Bardin 12b7dac124
Merge pull request #16833 from hashicorp/jbardin/plan-shutdown
Fully enable shutdown for plan and refresh in the local backend
2017-12-05 16:48:34 -05:00
Eyal Posener e1dadaae44 command: use newer version of "complete" library
This takes care of filtering results for us, so we don't need to do it on our end anymore.
2017-12-05 10:24:04 -08:00
James Bardin 2941ed464c replace the testShutdownHook with a check for Stop
Now that the local backend can be cancelled during plan and refresh, we
don't really need the testShutdownHook. Simplify the tests by just
checking for Stop being called on the provider.
2017-12-05 10:17:20 -05:00
James Bardin e2501d7830 make apply shutdown test completely deterministic
Add a shutdown hook to verify that a context has been correctly
cancelled, so we can remove the sleep and stop guessing.

Add a plan version of the shutdown test as well.
2017-12-01 15:56:49 -05:00
James Bardin 3aaa1e9d04 make plans cancellable
There was no cancellation context for a plan, so it would always have to
run to completion as SIGINT was being swallowed.

Move the shutdown channel to the command Meta since it's used in
multiple commands.
2017-12-01 13:14:44 -05:00
Martin Atkins ba0514106a return tfdiags.Diagnostics from validation methods
Validation is the best time to return detailed diagnostics
to the user since we're much more likely to have source
location information, etc than we are in later operations.

This change doesn't actually add any detail to the messages
yet, but it changes the interface so that we can gradually
introduce more detailed diagnostics over time.

While here there are some minor adjustments to some of the
messages to improve their consistency with terminology we
use elsewhere.
2017-11-28 11:15:29 -08:00
James Bardin 3c807e5427 update state test strings in command package 2017-11-07 21:18:08 -05:00
Martin Atkins 3b180d7f8a
Merge #16543: Bring the "terraform" provider back into core 2017-11-03 12:08:31 -07:00
Martin Atkins d4ee58ce59 Re-integrate the "terraform" provider into the main binary
As part of the 0.10 core/provider split we moved this provider, along with
all the others, out into its own repository.

In retrospect, the "terraform" provider doesn't really make sense to be
separated since it's just a thin wrapper around some core code anyway,
and so re-integrating it into core avoids the confusion that results when
Terraform Core and the terraform provider have inconsistent versions of
the backend code and dependencies.

There is no good reason to use a different version of the backend code
in the provider than in core, so this new "internal provider" mechanism
is stricter than the old one: it's not possible to use an external build
of this provider at all, and version constraints for it are rejected as
a result.

This provider is also run in-process rather than in a child process, since
again it's just a very thin wrapper around code that's already running
in Terraform core anyway, and so the process barrier between the two does
not create enough advantage to warrant the additional complexity.
2017-11-03 11:36:31 -07:00
James Bardin 964054efc0 update tests for updated output 2017-11-03 11:13:31 -04:00
James Bardin 1bf64ec788 add "Updating" output and fix output tests 2017-11-03 10:28:08 -04:00
James Bardin 9c334fe012 update init output
Change "Downloading" to 'Initializing" to match the provider loading
dialog.

List each module being loaded.

If a regisry module is being downloaded, list the registry host, and the
version discovered.

Show the source string from the config that is being fetched, rather
than the go-getter url. The full source can be found in the logs for
debugging.

Add much more extensive logging
2017-11-03 10:28:07 -04:00
James Bardin 523b121341 fix get test working directory
use a temp dir and cleanup
2017-11-02 15:38:53 -04:00
Martin Atkins 3da5fefdc1 command: Allow TF_DATA_DIR env var to override data directory
This allows the user to customize the location where Terraform stores
the files normally placed in the ".terraform" subdirectory, if e.g. the
current working directory is not writable.
2017-11-01 16:55:23 -07:00
Martin Atkins 400038eda4 command: "terraform apply" uses interactive confirmation by default
In the 0.10 release we added an opt-in mode where Terraform would prompt
interactively for confirmation during apply. We made this opt-in to give
those who wrap Terraform in automation some time to update their scripts
to explicitly opt out of this behavior where appropriate.

Here we switch the default so that a "terraform apply" with no arguments
will -- if it computes a non-empty diff -- display the diff and wait for
the user to type "yes" in similar vein to the "terraform destroy" command.

This makes the commonly-used "terraform apply" a safe workflow for
interactive use, so "terraform plan" is now mainly for use in automation
where a separate planning step is used. The apply command remains
non-interactive when given an explicit plan file.

The previous behavior -- though not recommended -- can be obtained by
explicitly setting the -auto-approve option on the apply command line,
and indeed that is how all of the tests are updated here so that they can
continue to run non-interactively.
2017-11-01 06:54:39 -07:00
James Bardin 53c8c1e208 e2e test for `init -from-module`
Pull down the hashicorp/vault/aws module into the current directory with
init.
2017-10-30 11:32:40 -04:00
James Bardin f64851242e use Storage.GetModule for init -from-module
This will enable copying any module source into a target directory.
2017-10-27 19:27:20 -04:00
James Bardin 5203c66116 pass command credentials into module.Storage 2017-10-27 16:16:35 -04:00
James Bardin 3a495ffe56 rename ModuleStorage to Storage
get rid of stutter and use module.Storage
2017-10-27 13:11:21 -04:00
James Bardin f2a7b94692 use the new ModuleStorage in the command package
Update the command package to use the new module storage. Move the old
command output strings into the module storage itself. This could be
moved back later either by using ui callbacks, or designing a module
storage interface once we know what the final requirements will look
like.
2017-10-27 12:58:24 -04:00
Martin Atkins 5347f82f9a command: Include provider versions in "terraform version"
We encourage users to share the "terraform version" output as part of
filing an issue, but previously it only printed the core Terraform version
and this left provider maintainers with no information about which
_provider_ version an issue relates to.

Here we make a best effort to show versions for providers, though we will
omit some or all of them if either "terraform init" hasn't been run (and
so no providers were selected yet) or if there are other inconsistencies
that would cause Terraform to object on startup and require a re-run of
"terraform init".
2017-10-25 17:36:24 -07:00
Martin Atkins 34cecfa839 command/e2etest: fix incorrect "terraform version" test
Two different errors here caused this test to pass even though it was
incorrect: the wanted version string was incorrect, but the test for it
was also inverted, and so together this made the test pass even though
it was actually not testing the output at all.
2017-10-25 17:36:24 -07:00
James Bardin 47c45788bc Merge pull request #16406 from hashicorp/jbardin/version-package
Add a standalone version package for all of terraform
2017-10-20 21:53:42 -04:00
Patrick Van Stee a4a53f6f99 command/validate: Load plugins from plugin_path file
This was added to other commands in 0.10 but was missed on "validate".
2017-10-20 15:42:51 -07:00
James Bardin 36b8be43e8 use the new version package
Update all references to the version values to use the new package.
The VersionString function was left in the terraform package
specifically for the aws provider, which is vendored. We can remove that
last call once the provider is updated.
2017-10-19 21:48:08 -04:00
Martin Atkins 865e61b4ea main+command: provide service disco and creds to commands
The command package is the main place we need access to these, so that
we can use them during init (to install packages, for example) and so that
we can use them to configure remote backends.

For the moment we're just providing an empty credentials object, which
will start to include both statically-configured and
helper-program-provided credentials sources in subsequent commits.
2017-10-19 11:18:43 -07:00
Martin Atkins 6f7bc4f5d7 command: use c.showDiagnostics for backend operation errors
This allows richer diagnostics produced by some subsystems to be displayed
in full-fidelity to the user.
2017-10-16 17:53:06 -07:00
Martin Atkins 5cd00a13ec command: use new diagnostics output for config errors
This uses the new diagnostics printer for config-related errors in the
main five commands that deal with config.

The immediate motivation for this is to allow HCL2-produced diagnostics
to be printed out in their full fidelity, though it also slightly changes
the presentation of other errors so that they are not presented in all
red text, which can be hard to read on some terminals.
2017-10-06 11:46:07 -07:00
Martin Atkins ea81e75a4e command: utility for rendering tfdiag diagnostics
This new method showDiagnostics takes any value that would be accepted by
tfdiags.Append and renders it to the UI.

This is intended to encourage consistent handling of the different kinds
of errors and diagnostics that can be produced, and allow richer error
objects like the HCL2 diagnostics to be easily unwrapped and shown in
their full-fidelity.
2017-10-06 11:46:07 -07:00
Martin Atkins fc20f419dd config and command: use errwrap to propagate config load errors
Previously we were using fmt.Sprintf and thus forcing the stringification
of the wrapped error.

Using errwrap allows us to unpack the original error at the top of the
stack, which is useful when the wrapped error is really a hcl.Diagnostics
containing potentially-multiple errors and possibly warnings.
2017-10-06 11:46:07 -07:00
Martin Atkins 86343fa895 Remove test output file accidentally added in 52c97e9f
Also, fix the missing gitignore rule that caused this to be added in the
first place.
2017-10-03 17:32:26 -07:00
James Bardin 91442b7146 Merge pull request #15680 from brunomcustodio/etcdv3-backend
[WIP] etcd v3 backend with lock support.
2017-10-03 14:15:58 -04:00
Martin Atkins 30e6f4e066 command/e2etest: test installation of plugins from cache
This is a tough one to unit tests because the behavior is tangled up in
the code that hits releases.hashicorp.com, so we'll add this e2etest as
some extra insurance that this works end-to-end.
2017-09-29 14:03:09 -07:00
Martin Atkins 5c0670fdf4 command: allow callers to activate a plugin cache
Here we add a new caller-settable field to command.Meta that activates
a read-through cache directory for plugin installation.
2017-09-29 14:03:09 -07:00
Martin Atkins 73d1298572 command/e2etest: test the "running in automation" workflow
Since we now have a guide that recommends some specific ways to run
Terraform in automation, we can mimic those suggestions in an e2e test and
thus ensure they keep working.

Here we test the three different approaches suggested in the guide:
- init, plan, apply (main case)
- init, apply (e.g. for deploying to a QA/staging environment)
- init, plan (e.g. for verifying a pull request)
2017-09-28 14:35:51 -07:00
Martin Atkins cb6d4e5f20 command/e2etest: fix TestPrimarySeparatePlan test
In 6712192724 we stopped counting data
source destroys in the destroy tally since they are an implementation
detail.

This caused this test to start failing, though since the new behavior is
correct here we just update the test to match.
2017-09-28 14:35:51 -07:00
Martin Atkins ece06c35b8 command: parameter autocomplete for "terraform workspace ..."
Shell tab completion for all of the subcommands under
"terraform workspace", providing the appropriate kind of auto-complete for
each argument, along with completion for for any flags.
2017-09-26 14:01:13 -07:00
Martin Atkins 793da43a72 command: workspace name autocomplete helper
This helper is a Predictor for the "complete" package that tries to
auto-complete workspace names from the current backend, if it's
initialized and operable.
2017-09-26 14:01:13 -07:00
Martin Atkins e4f18c3f4d command: multi-argument autocomplete helper
The predictors built in to the "complete" package assume that the same
type of argument is repeated indefinitely, but most Terraform commands
don't work like that, so this helper allows us to define a sequence of
predictors that apply to each argument in turn.
2017-09-26 14:01:13 -07:00
Martin Atkins 2ed9aa6077 command: autocomplete data for "terraform init" 2017-09-26 14:01:13 -07:00
Martin Atkins 82fefbc599 command: add some common autocomplete predictors
We use boolean flags and module sources in a lot of places, so we'll
define global predictors for these which we can use across many commands.
2017-09-26 14:01:13 -07:00
Sunny 8928c5a60a command/state-rm: tell the user how many items were removed 2017-09-20 12:48:27 -07:00
Jack Bruno 3f2136d7ee cli: terraform import -ignore-missing-config
This new option allows importing without configuration present.

Configuration is required by default as a confirmation that the provided resource name is correct, but it can be useful to override this in tools that wrap Terraform to do more involved operations.
2017-09-18 11:41:30 -07:00
Jearvon Dharrie 5afe1d39d1 Add `/downloads` to the outdated version message 2017-09-15 12:58:40 -07:00
Martin Atkins 0fe43c8977 cli: allow disabling "next steps" message in terraform plan
In #15884 we adjusted the plan output to give an explicit command to run
to apply a plan, whereas before this command was just alluded to in the
prose.

Since releasing that, we've got good feedback that it's confusing to
include such instructions when Terraform is running in a workflow
automation tool, because such tools usually abstract away exactly what
commands are run and require users to take different actions to
proceed through the workflow.

To accommodate such environments while retaining helpful messages for
normal CLI usage, here we introduce a new environment variable
TF_IN_AUTOMATION which, when set to a non-empty value, is a hint to
Terraform that it isn't being run in an interactive command shell and
it should thus tone down the "next steps" messaging.

The documentation for this setting is included as part of the "...in
automation" guide since it's not generally useful in other cases. We also
intentionally disclaim comprehensive support for this since we want to
avoid creating an extreme number of "if running in automation..."
codepaths that would increase the testing matrix and hurt maintainability.

The focus is specifically on the output of the three commands we give in
the automation guide, which at present means the following two situations:

* "terraform init" does not include the final paragraphs that suggest
  running "terraform plan" and tell you in what situations you might need
  to re-run "terraform init".
* "terraform plan" does not include the final paragraphs that either
  warn about not specifying "-out=..." or instruct to run
  "terraform apply" with the generated plan file.
2017-09-14 10:51:41 -07:00
Martin Atkins b95610c173 Merge #15387: terraform fmt -check option
This new option causes fmt to return a non-zero exit code if any inconsistencies are detected.
2017-09-14 09:51:10 -07:00
Martin Atkins fe4cfd03b5 command/format: restore "(forces new resource)" caption
In 3ea1592 the plan rendering was refactored to add an extra indirection
of producing a display-oriented plan object first and then rendering from
that object.

There was a logic error while adapting the existing plan rendering code
to use the new display-oriented object: the core InstanceDiff object sets
the "Destroy" flag (a boolean) for both DiffDestroy and DiffDestroyCreate,
and so this code previously checked r.Destroy to recognize the
"destroy-create" case. This was incorrectly adapted to a check for the
display action being DiffDestroy, when it should actually have been
DiffDestroyCreate.

The effect of this bug was to cause the "(forces new resource)"
annotations to not be displayed on attributes, though the resource-level
information still correctly reflected that a new resource was required.

This fix restores the attribute-level annotations.
2017-09-11 12:55:32 -07:00
Bruno Miguel Custodio 52c97e9fc9
First steps towards an 'etcdv3' backend. 2017-09-08 09:43:33 +01:00
Martin Atkins 83414beb8f command: various adjustments to the diff presentation
The previous diff presentation was rather "wordy", and not very friendly
to those who can't see color either because they have color-blindness or
because they don't have a color-supporting terminal.

This new presentation uses the actual symbols used in the plan output
and tries to be more concise. It also uses some framing characters to
try to separate the different stages of "terraform plan" to make it
easier to visually navigate.

The apply command also adopts this new plan presentation, in preparation
for "terraform apply" (with interactive plan confirmation) becoming the
primary, safe workflow in the next major release.

Finally, we standardize on the terminology "perform" and "actions" rather
than "execute" and "changes" to reflect the fact that reading is now an
action and that isn't actually a _change_.
2017-09-01 17:55:05 -07:00
Martin Atkins d4efc95191 command: show resource actions using resource addresses
Previously we were using the internal resource id syntax in the UI. Now
we'll use the standard user-facing resource address syntax instead.
2017-09-01 17:55:05 -07:00
Martin Atkins 3ea159297c command/format: improve consistency of plan results
Previously the rendered plan output was constructed directly from the
core plan and then annotated with counts derived from the count hook.
At various places we applied little adjustments to deal with the fact that
the user-facing diff model is not identical to the internal diff model,
including the special handling of data source reads and destroys. Since
this logic was just muddled into the rendering code, it behaved
inconsistently with the tally of adds, updates and deletes.

This change reworks the plan formatter so that it happens in two stages:
- First, we produce a specialized Plan object that is tailored for use
  in the UI. This applies all the relevant logic to transform the
  physical model into the user model.
- Second, we do a straightforward visual rendering of the display-oriented
  plan object.

For the moment this is slightly overkill since there's only one rendering
path, but it does give us the benefit of letting the counts be derived
from the same data as the full detailed diff, ensuring that they'll stay
consistent.

Later we may choose to have other UIs for plans, such as a
machine-readable output intended to drive a web UI. In that case, we'd
want the web UI to consume a serialization of the _display-oriented_ plan
so that it doesn't need to re-implement all of these UI special cases.

This introduces to core a new diff action type for "refresh". Currently
this is used _only_ in the UI layer, to represent data source reads.
Later it would be good to use this type for the core diff as well, to
improve consistency, but that is left for another day to keep this change
focused on the UI.
2017-09-01 17:55:05 -07:00
Sunny 3a1582c1b9 command/validate: read terraform.tfvars file for variable values
This is now consistent with the handling of this file for other commands.
2017-08-28 12:01:11 -07:00
Sunny 2d849f8650 command/init: check required_version
Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.
2017-08-28 11:25:16 -07:00
Martin Atkins c12d64f340 Use t.Helper() in our test helpers
Go 1.9 adds this new function which, when called, marks the caller as
being a "helper function". Helper function stack frames are then skipped
when trying to find a line of test code to blame for a test failure, so
that the code in the main test function appears in the test failure output
rather than a line within the helper function itself.

This covers many -- but probaly not all -- of our test helpers across
various packages.
2017-08-28 09:59:30 -07:00
Radek Simko 4e8ccb3127 Merge pull request #15828 from hashicorp/e2e-decoupling
tools/terraform-bundle: Add e2e tests
2017-08-22 18:33:59 +02:00
Radek Simko aabab16069
command/init: Remove extra whitespace 2017-08-17 09:43:09 +02:00
Michael Handler 817d1c4869 Output duration in UiHook after successful operation completion. 2017-08-16 14:41:41 -07:00
James Bardin 18af7750d4 Merge pull request #14720 from sevagh/master
Add salt-masterless provisioner
2017-08-16 16:47:46 -04:00
Radek Simko 9e7e4ff4fb
e2e: Decouple logic for running e2e tests 2017-08-16 18:20:13 +02:00
Radek Simko 28b33c9299
plugin: Display version + source when initializing plugins 2017-08-15 19:29:13 +02:00
Radek Simko 38cb98fb0c
Comply w/ latest changes in mitchellh/cli 2017-08-14 17:19:40 +02:00
James Bardin ea3e87b584 Merge pull request #15768 from hashicorp/jbardin/remote-import
Don't ForceLocal for the import backend
2017-08-11 09:12:46 -04:00
James Bardin 5bcc1bae59 Merge pull request #15769 from hashicorp/jbardin/discovery-paths
load legacy plugin paths
2017-08-09 17:54:59 -04:00
James Bardin 2bb5007690 Merge pull request #15683 from hashicorp/jbardin/remote-state-lineage
Remove strict lineage check in remote.State
2017-08-09 17:49:19 -04:00
James Bardin 37932c3cd6 make state_rm flag description match state_mv 2017-08-09 14:03:40 -04:00
James Bardin 4034f988df update import command docs
Fix the -state and -state-out wording to be consistent with other
commands. Remove the erroneous reference to remote state in the website
version of the flag description.
2017-08-09 14:01:45 -04:00
James Bardin 54998933f5 load Meta.PluginOverrides in dicovery
Make sure the override paths from the legacy rc file are loaded for
discovery.
2017-08-09 11:14:33 -04:00
James Bardin fa20d43d80 test loading of Meta.PluginOverrides
These are currently being skipped in discovery
2017-08-09 11:13:54 -04:00
James Bardin e3748901b4 Don't ForceLocal for the import backend
While the `local.Local` backend is the only implementation of
`backend.Local`, creating the backend with `ForceLocal` bypasses the
`backend.Backend` in the `local.Local` causing a local state to be
implicitly created rather than using the configured state backend.

Add a test that imports into a configured backend (using the "local"
backend as a remote state proxy). This further confirms the confusing
nature of ForceLocal, as the backend _is_ local, but not from the
viewpoint of meta.Backend.
2017-08-09 10:24:32 -04:00
Sevag Hanssian 867760ed56 Add salt-masterless provisioner 2017-08-07 10:00:29 -04:00
James Bardin eadda50f02 Merge pull request #15652 from hashicorp/jbardin/state-command
state commands with remote state backends
2017-08-03 13:29:45 -04:00
James Bardin 55d18dcef2 update state rm amd mv docs
Update the documentation to match the current behavior, and make the
usage output and website docs match.
2017-08-03 13:24:23 -04:00
James Bardin 07b0101fb5 update workspace new test for inmem backend
The existing test assumed local state files.
2017-08-01 19:34:22 -04:00
James Bardin 16e8e405c7 create failing test cases for remote lineage issue
Some remote backend would fail on `-state push -force`, or `workspace
new -state` because of a new strict lineage check in remote.State.
2017-08-01 19:34:21 -04:00
Martin Atkins 8a7a0a7459 command: terraform init -from-module=...
This restores the earlier behavior of the first positional argument to
terraform init in 0.9, but as a command line option.

The positional argument was removed to improve consistency with other
commands that take a working directory as their first positional argument.
It was originally intended that this functionality would return in a
later release along with some other general improvements to Terraform's
module handling, but we're introducing here an interim solution that
uses the existing module source concept, to allow for easier porting of
workflows that previously depended on the automatic copy behavior.

In a future release this feature may change again as the module
improvements design firms up, but we expect it to be broadly compatible
with this temporary state.
2017-07-28 15:23:29 -07:00
James Bardin 51547ad2ba add tests for state commands through a backend 2017-07-27 18:06:47 -04:00
James Bardin 5b4ae36cb0 don't print help for state loading errors
These already include detailed messages, and it's not a usage issue,
it's a config or file location issue.
2017-07-27 18:06:47 -04:00
James Bardin 33ba6774e0 Make the state commands use the real command.Meta
In order to use a backend for the state commands, we need an initialized
meta. Use a single Meta instance rather than temporary ones to make sure
the backends are initialized properly.
2017-07-27 15:33:50 -04:00
James Bardin 45a9edb763 make state mv and rm work with remote states
The default value for the -state flag was overriding the location of any
remote state.
2017-07-27 09:41:39 -04:00
Kyle McCullough ad896b65c9
command: add -check flag to fmt (#15304) 2017-07-21 14:37:15 -05:00
James Bardin 0bca383b6b Merge pull request #15528 from alrs/command-swallowed-errors
Fix swallowed errors in command package.
2017-07-20 08:57:10 -04:00
Lars Lehtonen 119996b815
Return immediately after newly-added error condition 2017-07-19 15:46:21 -07:00
James Bardin eb02467298 Merge pull request #15588 from hashicorp/jbardin/no-graph-config
graph should not panic with no config
2017-07-18 13:10:00 -04:00
James Bardin f10163ecc7 graph should not panic with no config
The backends replace a nil module tree with an empty one before building
the graph, so the graph command needs to do the same.
2017-07-18 13:03:57 -04:00
Martin Atkins 23f9c8785e command/e2etest: an initial test for the primary workflow
This e2etest runs an init, plan, apply, destroy sequence against a test
configuration using the real template and null providers downloaded from
the official repository.

This test _does_ trample a bit on the scope of some already-existing
tests, but this is mainly just to check our assumptions about how
Terraform behaves to ensure that we can reach our main conclusion here:
that the main Terraform workflow commands interact correctly with each
other in real use and we can complete the full workflow.
2017-07-17 14:25:33 -07:00
Martin Atkins 52df81ee49 command/e2etest: test that we can install provider plugins
We already have good tests for the business logic around provider
installation, but the existing tests all stub out the main repository
server. This test completes that coverage by verifying that the installer
is able to run against the real repository and install an official release
of the template provider.
2017-07-17 14:25:33 -07:00
Martin Atkins 0e0b0d125a command/e2etest: "terraform version" test
This basic test is here primarily because it's one of the few that can
run without reaching out to external services, and so it means our usual
test runs will catch situations where the main executable build is
somehow broken.

The version command itself is not very interesting to test, but it's
convenient in that its behavior is very predictable and self-contained.
2017-07-17 14:25:33 -07:00
Martin Atkins fee61a44b4 command/e2etest: end-to-end testing harness
Previously we had no automated testing of whether we can produce a
Terraform executable that actually works. Our various functional tests
have good coverage of specific Terraform features and whole operations,
but we lacked end-to-end testing of actual usage of the generated binary,
without any stubbing.

This package is intended as a vehicle for such end-to-end testing. When
run normally under "go test" it will produce a build of the main Terraform
binary and make it available for tests to execute. The harness exposes
a flag for whether tests are allowed to reach out to external network
services, controlled with our standard TF_ACC environment variable, so
that basic local tests can be safely run as part of "make test" while
more elaborate tests can be run easily when desired.

It also provides a separate mode of operation where the included script
make-archive.sh can be used to produce a self-contained test archive that
can be copied to another system to run the tests there. This is intended
to allow testing of cross-compiled binaries, by shipping them over to
the target OS and architecture to run without requiring a full Go compiler
installation on the target system.

The goal here is not to test again functionality that's already
well-covered by our existing tests, but rather to test chains of normal
operations against the build binary that are not otherwise tested
together.
2017-07-17 14:25:33 -07:00
Lars Lehtonen 8501d83e6c
Fix swallowed errors in command package. 2017-07-11 08:01:02 -07:00
James Bardin 583cc350a9 fix minor races in workspace tests
The improved err scanner loop in meta causes these to race. There's no
need to write back to the same commands struct, so just use a new
instance in each iteration.
2017-07-06 13:49:32 -04:00
James Bardin f7f1e8e406 Sort arguments in Meta.process
Meta.process was relying on the system readdir to order the arguments,
but readdir doesn't guarantee any ordering. Read the directory contents
as a whole and sort them in place before adding the tfvars files.
2017-07-06 11:34:47 -04:00
James Bardin 0fdcf2c01e properly cleanup and print correct error messages
Some Meta tests were not cleaning up their temp directories.

The process test wasn't printing the correct arguments in the error
messages.
2017-07-06 11:33:32 -04:00
Robert Liebowitz 4bf2269784 Add test case for autoloading tfvars 2017-07-05 17:24:17 -07:00
Robert Liebowitz e67ecb5ce2 Restructure Meta.process to logically group code 2017-07-05 17:24:17 -07:00
Robert Liebowitz 8d98fdecac Autoload only .auto.tfvars files 2017-07-05 17:24:17 -07:00
Robert Liebowitz 006744bfe0 Use all tfvars files in working directory
As a side effect, several commands that previously did not have a failure
state can now fail during meta-parameter processing.
2017-07-05 17:24:17 -07:00
Martin Atkins 39c4d6ab1f Merge #15424: Improve robustness of state persistence handling
Previously the APIs for state persistence and management had some problematic cases where we depended on hidden mutations of the state structure as side-effects of otherwise-innocent-looking operations, which was a frequent cause of accidental regressions due to faulty assumptions.

This new model attempts to isolate certain state mutations to just within the state managers, and makes the state managers work on separated snapshots of the state rather than on the "live" object to reduce the risk of race conditions.
2017-07-05 16:27:08 -07:00
Martin Atkins fee1197cf9 command: terraform state rm to require at least one argument
Due to how the state filter machinery works, passing no arguments is valid
and matches _all_ resources.

It is very unlikely that someone wants to remove everything from state, so
this ends up being a very dangerous default for the "terraform state rm"
command, and surprising for someone who perhaps runs it looking for the
usage information.

So we'll be pragmatic here and reject the no-arguments case for this
command, accepting that it makes the unlikely case of intentionally
deleting all resources harder in order to make it less likely that it
will happen _unintentionally_.

If someone does really want to remove all resources from the state, they
can provide an explicit empty string argument, but this isn't documented
because it's a weird case that doesn't seem worth mentioning.

This fixes #15283.
2017-07-05 16:19:32 -07:00
James Bardin fb397060eb add some Serial checks to apply and refresh tests 2017-07-05 18:18:28 -04:00
James Bardin 054716c397 use testStateRead helper in apply tests
We have a helper function that we hardly ever use.
TODO: convert the rest of the manual ReadState calls eventually.
2017-07-05 18:18:16 -04:00
James Bardin 501cbeaffe testState shouldn't rely on mods from WriteState
The state returned from the testState helper shouldn't rely on any
mutations caused by WriteState. The Init function (which is analogous to
NewState) shoudl set any required fields.
2017-07-05 17:47:05 -04:00
Mike Helmick 9d7fce2f69 command: "terraform workspace show" to print current workspace name
This command serves as an alternative to the human-oriented list of workspaces for scripting use-cases where it's useful to know the _current_ workspace name.
2017-07-05 14:35:46 -07:00
Martin Atkins 4d53eaa6df state: more robust handling of state Serial
Previously we relied on a constellation of coincidences for everything to
work out correctly with state serials. In particular, callers needed to
be very careful about mutating states (or not) because many different bits
of code shared pointers to the same objects.

Here we move to a model where all of the state managers always use
distinct instances of state, copied when WriteState is called. This means
that they are truly a snapshot of the state as it was at that call, even
if the caller goes on mutating the state that was passed.

We also adjust the handling of serials so that the state managers ignore
any serials in incoming states and instead just treat each Persist as
the next version after what was most recently Refreshed.

(An exception exists for when nothing has been refreshed, e.g. because
we are writing a state to a location for the first time. In that case
we _do_ trust the caller, since the given state is either a new state
or it's a copy of something we're migrating from elsewhere with its
state and lineage intact.)

The intent here is to allow the rest of Terraform to not worry about
serials and state identity, and instead just treat the state as a mutable
structure. We'll just snapshot it occasionally, when WriteState is called,
and deal with serials _only_ at persist time.

This is intended as a more robust version of #15423, which was a quick
hotfix to an issue that resulted from our previous slopping handling
of state serials but arguably makes the problem worse by depending on
an additional coincidental behavior of the local backend's apply
implementation.
2017-07-05 12:34:30 -07:00
Radek Simko 14614a5423 command/validate: Add flag to check that all variables are specified (#13872)
* command/validate: Add flag to check that all variables are specified

* Rename config-only to check-variables
2017-07-05 17:32:29 +01:00
Jake Champlin 9944ea6886
core: Skip provider checksum validation based on env var
Skips checksum validation if the `TF_SKIP_PROVIDER_VERIFY` environment variable is set. Undocumented variable, as the primary goal is to significantly improve the local provider development workflow.
2017-07-03 13:59:13 -04:00
James Bardin da1c45b566 Merge pull request #15412 from hashicorp/jbardin/windows-delete-workspace
Unlock state before workspace deletion
2017-06-27 17:47:34 -04:00
James Bardin 1c78dfb286 Unlock state before workspace deletion
We need to release the lock just before deleting the state, in case the backend
can't remove the resource while holding the lock. This is currently true for
Windows local files.

TODO: While there is little safety in locking while deleting the state, it
might be nice to be able to coordinate processes around state deletion, i.e. in
a CI environment. Adding Delete() as a required method of States would allow
the removal of the resource to be delegated from the Backend to the State
itself.
2017-06-27 14:31:08 -04:00
David Glasser 14af879fe0 command: also print plan for destroy 2017-06-27 11:22:31 -07:00
David Glasser 039d36bf91 command: add "apply -auto-approve=false" flag
A common reason to want to use `terraform plan` is to have a chance to
review and confirm a plan before running it.  If in fact that is the
only reason you are running plan, this new `terraform apply -auto-approve=false`
flag provides an easier alternative to

    P=$(mktemp -t plan)
    terraform refresh
    terraform plan -refresh=false -out=$P
    terraform apply $P
    rm $P

The flag defaults to true for now, but in a future version of Terraform it will
default to false.
2017-06-27 11:22:26 -07:00
James Bardin 22074f1029 Merge pull request #15408 from hashicorp/jbardin/import-plugin-dir
Make sure import uses complete plugin path
2017-06-27 13:07:47 -04:00
Patrick Aikens 7775ef809b Display correct to and from backends in copy message when migrating to new remote state (#15318) 2017-06-27 19:05:45 +03:00
James Bardin 85b9069eb7 test import with -plugin-dir
The import command wasn't loading the full plugin path for discovery.
Run a basic plugin init sequence, and verify we can find a plugin (even
though the plugin is invalid and will fail).
2017-06-27 11:07:45 -04:00
James Bardin 0cbd51c365 find -plugin-dir during import
The import command wasn't loading the plugin path at all, relying on the
local directory for binaries.

Load the plugin dir into Meta, and pass in ForceLocal for consistency.
The Backend returned was going to be a Local anyway, so the added check
wasn't ensuring anything.
2017-06-26 18:28:45 -04:00
Martin Atkins 3df164502a command: don't prompt for state migration if TF_INPUT is set
The "confirm" method was directly checking the meta struct's input field,
but that only represents the -input command line flag, and doesn't
respect the TF_INPUT environment variable.

By calling the Input method instead, we check both.

This fixes #15338.
2017-06-26 11:22:29 -07:00
James Bardin 833cc9a6c5 Fix state mv/rm -backup documentation
There is only one backup made, contrary to the help text. We may want to
implement that, but the documentation should match the current behavior.
2017-06-23 14:46:09 -04:00
James Bardin 6e7baaaeff don't load the backend when -state is provided
When using a `state` command, if the `-state` flag is provided we do not
want to modify the Backend state. In this case we should always create a
local state instance.

The backup flag was also being ignored, and some tests were relying on
that, which have been fixed.
2017-06-23 14:41:49 -04:00
James Bardin 5f939b42fe test that `state mv -state` doesn't use Backend
If we provide a -state flag to a state command, we do not want terraform
to modify the backend state. This test fails since the state specified
in the backend doesn't exist
2017-06-23 14:39:37 -04:00
James Bardin 4893fcc24f improve plugin error formatting
Remove "checksum" from the error, and only indicate that the plugin has
changed.

Always show requested versions even if it's "any", and found versions of
plugins.
2017-06-22 15:31:36 -04:00
James Bardin 103ab20b00 add required and found versions to error output
Provide the user with feedback showing the version requirements and the
versions found when there's is a plugin error
2017-06-22 14:20:50 -04:00
James Bardin 7a955f990c make display plugin checksum error to user
The error follows a generic message, so can be ignored by users who may
not understand the implications.
2017-06-22 13:38:55 -04:00
Martin Atkins 0dc6d97a37 command/format: minor adjustments to plan rendering
This change makes various minor adjustments to the rendering of plans
in the output of "terraform plan":

- Resources are identified using the standard resource address syntax,
  rather than exposing the legacy internal representation used in the
  module diff resource keys. This fixes #8713.

- Subjectively, having square brackets in the addresses made it look more
  visually "off" when the same name but with different indices were
  shown together with differing-length "symbols", so the symbols are now
  all padded and right-aligned to three characters for consistent layout
  across all operations.

- The -/+ action is now more visually distinct, using several different
  colors to help communicate what it will do and including a more obvious
  "(new resource required)" marker to help draw attention to this not
  being just an update diff. This fixes #15350.

- The resources are now sorted in a manner that sorts index [10] after
  index [9], rather than after index [1] as we did before. This makes it
  easier to scan the list and avoids the common confusion where it seems
  that there are only 10 items when in fact there are 11-20 items with
  all the tens hiding further up in the list.
2017-06-22 07:03:23 -07:00
Martin Atkins 40e2fbb8e9 command: init to allow plugin init without backend init
Previously init would crash if given these options:
    -backend=false -get-plugins=true

This is because the state is used as a source of provider dependency
information, and we need to instantiate the backend to get the state.

To avoid the crash, we now use the following adjusted behavior:

- if -backend=true, we behave as before

- if -backend=false, we instead try to instantiate the backend the same
  way any other command would, without modifying its configuration

- if we're able to instantiate the backend, we use it to fetch state
  for dependency resolution purposes

- if the backend is not instantiable then we assume it's not yet
  configured and proceed with a nil state, which may cause us to see an
  incomplete picture of the dependencies but still allows the install
  to succeed. Subsequently running "terraform plan" will not work until
  the backend is (re-)initialized, so the incomplete picture of required
  plugins is safe.
2017-06-21 11:43:28 -07:00
Martin Atkins 3dc7d618f7 command: avoid empty string constraints in plugin install errors
This takes care of a few dangling cases where we were still stringifying
empty version constraints, which creates confusing error messages due to
it stringing as the empty string.

For the "no suitable versions available" message, we fall back on the
"provider not found" message if no versions were found even though it's
unconstrained. This should only happen in an edge case where the
provider's index page exists on the releases server but no versions are
yet present.

For the message about plugin protocol versions, this again is an edge
case since with no constraints this should happen only if we release
an incompatible Terraform version but don't release a new version of the
plugin that's compatible. In this case we just show the constraint as
"(any version)" to make sure we always show _something_.
2017-06-21 09:18:17 -07:00
James Bardin c10f5caf05 Merge pull request #15345 from hashicorp/jbardin/plugin-sig
verify automatically downloaded plugins
2017-06-20 17:05:54 -04:00
Martin Atkins 9c2fe3456b command: purge unused plugins as a side effect of plugin installation
Previously we only did this when _upgrading_, but that's unnecessarily
specific and confusing since e.g. plugins can get upgraded implicitly by
constraint changes, which would not then trigger the purge process.

Instead, we'll assume that the user is able to easily re-download plugins
that were purged here, or if they need more specific guarantees they will
manage manually a plugin directory and disable the auto-install behavior
using `-plugin-dir`.
2017-06-20 13:40:03 -07:00
Martin Atkins d48dcbb4a5 command: more-helpful error messages from plugin installation
Now we are able to recognize and handle a few special error situations
from plugin installation with more verbose error messages that give the
user better feedback on how to proceed.
2017-06-20 13:39:45 -07:00
James Bardin 0a47228065 add -verify-plugins flag to init
This provides the user with the option to disable plugin signature
verification.
2017-06-20 13:14:31 -04:00
James Bardin 6728c48d8c fix whitespace around backend init success message
make it better match the surrounding messages.
2017-06-19 12:08:42 -04:00
James Bardin da385c4268 Don't show plugin init message if there are none
Skip the message and plugin initialization if there are none in the
config.
2017-06-19 12:08:42 -04:00
James Bardin 2e57d284cb Merge pull request #15323 from hashicorp/jbardin/constraint-suggestion
remove "~> 0.0" constraint suggestions
2017-06-16 18:20:24 -04:00
James Bardin ec99b6910b remove "~> 0.0" constraint suggestions
Don't suggest constraints when the available plugin isn't versioned.

Add zero version const for comparisons.
2017-06-16 16:25:36 -04:00
James Bardin 270eedd4b8 always pass in the full plugin path to dicovery
Discovery no longer tries to walk into OS_ARCH dirs, so always pass in
the full search path.
2017-06-16 14:09:47 -04:00
James Bardin be2069ac81 add -plugin-dir option
The -plugin-dir option lets the user specify custom search paths for
plugins. This overrides all other plugin search paths, and prevents the
auto-installation of plugins.

We also make sure that the availability of plugins is always checked
during init, even if -get-plugins=false or -plugin-dir is set.
2017-06-15 15:23:16 -04:00
James Bardin 000e860706 Add plugin dir scaffolding
add pluginDir to command.Meta, the flag to initialize it, and the
methods to save and restore it.
2017-06-15 14:26:12 -04:00
James Bardin f723270e3e search the vendor directory for plugins
The default location for users to manually add plugins will be
./terraform.d/plugins/
2017-06-15 10:12:00 -04:00
James Bardin 4f5e92e4c0 reverse init test to check for dataDir in PWD
init should always write intternal data to the current directory, even
when a path is provided. The inherited behavior no longer applies to the
new use of init.
2017-06-14 15:22:30 -04:00
James Bardin 55bf19e548 always write to dataDir in the current directory
Now that init can take a directory for configuration, the old behavior
of writing the .terraform data directory into the target path no longer
makes sense. Don't change the dataDir field during init, and write to
the default location.

Clean up all references to Meta.dataDir, and only use the getter method
in case we chose to dynamically override this at some point.
2017-06-14 15:14:26 -04:00
Martin Atkins 6979a07754 command: init -upgrade for provider plugins
Now when -upgrade is provided to "terraform init" (and plugin installation
isn't disabled) it will:

- ignore the contents of the auto-install plugin directory when deciding
  what is "available", thus causing anything there to be reinstalled,
  possibly at a newer version.
- if installation completes successfully, purge from the auto-install
  plugin directory any plugin-looking files that aren't in the set of
  chosen plugins.

As before, plugins outside of the auto-install directory are able to
take precedence over the auto-install ones, and these will never be
upgraded nor purged.

The thinking here is that the auto-install directory is an implementation
detail directly managed by Terraform, and so it's Terraform's
responsibility to automatically keep it clean as plugins are upgraded.

We don't yet have the -plugin-dir option implemented, but once it is it
should circumvent all of this behavior and just expect providers to be
already available in the given directory, meaning that nothing will be
auto-installed, -upgraded or -purged.
2017-06-13 12:28:07 -07:00
Martin Atkins f753974bb3 plugin/discovery: Installer interface, and provider implementation
Previously we had a "getProvider" function type used to implement plugin
fetching. Here we replace that with an interface type, initially with
just a "Get" function.

For now this just simplifies the interface by allowing the target
directory and protocol version to be members of the struct rather than
passed as arguments.

A later change will extend this interface to also include a method to
purge unused plugins, so that upgrading frequently doesn't leave behind
a trail of unused executable files.
2017-06-13 12:28:07 -07:00
Gavin Williams 5834333ea3 command: terraform get -upgrade
As of this commit this just upgrades modules, but this option will also
later upgrade plugins and indeed anything else that's being downloaded and
installed as part of the init.
2017-06-13 12:28:07 -07:00
James Bardin be900e8085 Merge pull request #15258 from hashicorp/jbardin/remove-providers
Remove providers from core
2017-06-12 14:13:11 -04:00
James Bardin cbbbcea4b9 stop err scanner goroutines between tests 2017-06-12 13:52:30 -04:00
James Bardin 77a32f3df0 remove "core" distinction
Since there is little left that isn't core, remove the distinction for
now to reduce confusion, since a "core" binary will mostly work except
for provisioners.
2017-06-12 13:43:54 -04:00
James Bardin 7e7f53954d remove internal provider test from command pkg 2017-06-12 13:43:52 -04:00
James Bardin 81ac0ed204 re-generate plugin list 2017-06-12 13:42:07 -04:00
Radek Simko 4e85f552e7 Merge pull request #15246 from hashicorp/b-fix-cmd-provider-crash
command/providers: Avoid crash when no configs found
2017-06-12 15:48:55 +01:00
James Bardin a30007b41d Merge pull request #15032 from hashicorp/jbardin/init
change init args to remove source copy
2017-06-12 09:06:32 -04:00
Radek Simko f5ff67c530
command/providers: Add regression test 2017-06-10 12:02:01 +01:00
Radek Simko ee30df3efd
command/providers: Enable processing of meta-parameters
This will enable proper colouring of the output, like for other commands.
2017-06-10 11:51:06 +01:00
Radek Simko 86fbcfac83
command/providers: Avoid crash when no configs found 2017-06-10 11:51:03 +01:00
Martin Atkins f7ce6a15f8 backend: Operation.Environment renamed to "Workspace"
This is part of an effort to switch this terminology across all of
Terraform.
2017-06-09 16:26:26 -07:00
Martin Atkins 418a8a8bc9 command + backend: rename various API objects to "Workspace" terminology
We're shifting terminology from "environment" to "workspace". This takes
care of some of the main internal API surface that was using the old
terminology, though is not intended to be entirely comprehensive and is
mainly just to minimize the amount of confusion for maintainers as we
continue moving towards eliminating the old terminology.
2017-06-09 16:26:25 -07:00
Martin Atkins 5f9f13ab8f command: show warnings from backend config validation
Previously we just silently ignored warnings from validating the backend
config, but now that we have a deprecated argument it's important to print
these out so users can respond to the deprecation warning.
2017-06-09 15:01:39 -07:00
Martin Atkins 31d556894f command: shallow UI-focused rename of "environment" to "workspace"
Feedback after 0.9 was that the term "environment" was confusing due to
it colliding with several other concepts, such as OS environment
variables, a non-aligned Terraform Enterprise concept, and differing ideas
of "environment" within various organizations.

This new term "workspace" is intended to ease some of that confusion. This
term is not used anywhere else in Terraform today, and we expect it to not
be used in a manner that would be confusing within user organizations.

This begins a deprecation cycle for the "terraform env" family of commands,
instead moving to an equivalent set of "terraform workspace" commands.

There are some remaining references to the old "environment" concept in
the code, which will be cleaned up in a separate change. This change is
instead focused on text visible in the UI and wording within code comments
for the benefit of human maintainers of the code.
2017-06-09 15:01:39 -07:00
David Glasser c25d848ffb core: allow overriding environment name via env var
This allows you to run multiple concurrent terraform operations against
different environments from the same source directory.

Fixes #14447.

Also removes some dead code which appears to do the same thing as the function I
modified.
2017-06-09 15:01:39 -07:00
James Bardin 2bfcdbbd08 change init args to remove source copy
When init was modified in 0.9 to initialize a terraform working
directory, the legacy behavior was kept to copy or fetch module sources.
This left the init command without the ability that the plan and apply
commands have to target a specific directory for the operation.

This commit removes the legacy behavior altogether, and allows init to
target a directory for initialization, bringing it into parity with plan
and apply. If one want to copy a module to the target or current
directory, that will have to be done manually before calling init. We
can later reintroduce fetching modules with init without breaking this
new behavior, by adding the source as an optional second argument.

The unit tests testing the copying of sources with init have been
removed, as well as some out of date (and commented out) init tests
regarding remote states.
2017-06-09 17:50:14 -04:00
James Bardin d1c50efb01 change providers.json to lock.json
It might not just be for providers, and it's in the plugins dir, so
lock.json seems descriptive enough.
2017-06-09 14:03:59 -07:00
James Bardin 08592c2602 update plugin name in command test 2017-06-09 14:03:59 -07:00
James Bardin a529b64cc8 better init error output
Provide log-form message when a provider isn't found, along with the
desired constraints.
2017-06-09 14:03:59 -07:00
James Bardin fdbfc17fae missing constraints passed erroniously
ConstrainVersions was documented as returning nil, but it was instead
returning an empty set. Use the Count() method to check for nil or
empty. Add test to verify failed constraints will show up as missing.
2017-06-09 14:03:59 -07:00
Martin Atkins 766f8e5d64 command init: remove confusing uses of "environment" in the usage
"environment" is a very overloaded term, so here we prefer to use the
term "working directory" to talk about a local directory where operations
are executed on a given Terraform configuration.
2017-06-09 14:03:59 -07:00
Martin Atkins 3c429b3628 command init: show log output for each provider plugin downloaded
Each provider plugin will take at least a few seconds to download, so
providing feedback about each one should make users feel less like
Terraform has hung.

Ideally we'd show ongoing progress during the download, but that's not
possible without re-working go-getter, so we'll accept this as an interim
solution for now.
2017-06-09 14:03:59 -07:00
Martin Atkins 4571a16b15 command: remove Meta.forceProviderSHA256s
This was added with the idea of using it to override the SHA256 hashes
to match those hypothetically stored in a plan, but we already have a
mechanism elsewhere for populating context fields from plan fields, so
this is not actually necessary.
2017-06-09 14:03:59 -07:00
Martin Atkins 9aae06db97 command: update mockGetProvider.GetProvider for new interface
The expected type was changed in the mainline code but the tests were not
updated to match.
2017-06-09 14:03:59 -07:00
Martin Atkins 4ba20f9c1c command init: show suggested constraints for unconstrained providers
When running "terraform init" with providers that are unconstrained, we
will now produce information to help the user update configuration to
constrain for the particular providers that were chosen, to prevent
inadvertently drifting onto a newer major release that might contain
breaking changes.

A ~> constraint is used here because pinning to a single specific version
is expected to create dependency hell when using child modules. By using
this constraint mode, which allows minor version upgrades, we avoid the
need for users to constantly adjust version constraints across many
modules, but make major version upgrades still be opt-in.

Any constraint at all in the configuration will prevent the display of
these suggestions, so users are free to use stronger or weaker constraints
if desired, ignoring the recommendation.
2017-06-09 14:03:59 -07:00
James Bardin e0f2235f66 update init command with new GetProvider signature
GetProvider needs the plugin protocol version to be passed in
2017-06-09 14:03:59 -07:00
Martin Atkins 032f71f1ff command: produce provider lock file during "terraform init"
Once we've installed the necessary plugins, we'll do one more walk of
the available plugins and record the SHA256 hashes of all of the plugins
we select in the provider lock file.

The file we write here gets read when we're building ContextOpts to
initialize the main terraform context, so any command that works with
the context will then fail if any of the provider binaries change.
2017-06-09 14:03:59 -07:00
Martin Atkins 6ba6508ec9 command: pass the locked plugin hashes into ContextOpts
By reading our lock file and passing this into the context, we ensure that
only the plugins referenced in the lock file can be used. As of this
commit there is no way to create that lock file, but that will follow soon
as part of "terraform init".

We also provide a way to force a particular set of SHA256s. The main use
for this is to allow us to persist a set of plugins in the plan and
check the same plugins are used during apply, but it may also be useful
for automated tests.
2017-06-09 14:03:59 -07:00
Martin Atkins 720670fae7 command: helper to manage the provider plugins lock file
This is just a JSON file with the SHA256 digests of the plugin
executables.
2017-06-09 14:03:59 -07:00
Martin Atkins 7d0a98af46 command: provider resolver to also check SHA256 constraints when set
In addition to looking for matching versions, the caller can also
optionally require a specific executable by its SHA256 digest.
2017-06-09 14:03:59 -07:00
Martin Atkins e3401947a6 plugin/discovery: PluginRequirements can specify SHA256 digests
As well as constraining plugins by version number, we also want to be
able to pin plugins to use specific executables so that we can detect
drift in available plugins between commands.

This commit allows such requirements to be specified, but doesn't yet
specify any such requirements, nor validate them.
2017-06-09 14:03:59 -07:00
Martin Atkins 9a398a7793 command: require resource to be in config before import
Previously we encouraged users to import a resource and _then_ write the
configuration block for it. This ordering creates lots of risk, since
for various reasons users can end up subsequently running Terraform
without any configuration in place, which then causes Terraform to want
to destroy the resource that was imported.

Now we invert this and require a minimal configuration block be written
first. This helps ensure that the user ends up with a correlated resource
config and state, protecting against any inconsistency caused by typos.

This addresses #11835.
2017-06-09 14:03:59 -07:00
Martin Atkins 7d8719150c command: validate import resource address early
Previously we deferred validation of the resource address on the import
command until we were in the core guts, which caused the error responses
to be rather unhelpful.

By validating these things early we can give better feedback to the user.
2017-06-09 14:03:59 -07:00
Martin Atkins f6305fcc27 command: remove commented-out, misplaced tests
For some reason there was a block of commented-out tests for the refresh
command in the test file for the import command. Here we remove them to
reduce the noise in this file.
2017-06-09 14:03:59 -07:00
James Bardin 044ad5ef59 rename some Constraints methods per code review 2017-06-09 14:03:59 -07:00
James Bardin 2994c6ac5d add init getPlugin test
add a mock plugin getter, and test that we can fetch requested version
of the plugins.
2017-06-09 14:03:59 -07:00
James Bardin 4c32cd432a make getProvider pluggable 2017-06-09 14:03:59 -07:00
James Bardin 66ebff90cd move some more plugin search path logic to command
Make less to change when we remove the old search path
2017-06-09 14:03:59 -07:00
James Bardin 2749946f5c basic plugin getter
Add discovery.GetProviders to fetch plugins from the relases site.

This is an early version, with no tests, that only (probably) fetches
plugins from the default location. The URLs are still subject to change,
and since there are no plugin releases, it doesn't work at all yet.
2017-06-09 14:03:59 -07:00
James Bardin 7d2d951f27 Rename VersionSet to Constraints
VersionSet is a wrapper around version.Contraints, so rename it it as
such.
2017-06-09 14:03:59 -07:00
James Bardin 718ede0636 have Meta.Backend use a Config rather than loading
Instead of providing the a path in BackendOpts, provide a loaded
*config.Config instead. This reduces the number of places where
configuration is loaded.
2017-06-09 14:03:59 -07:00
Martin Atkins 3af0ecdf01 command: "terraform providers" command
This new command prints out the tree of modules annotated with their
associated required providers.

The purpose of this command is to help users answer questions such as
"why is this provider required?", "why is Terraform using an older version
of this provider?", and "what combination of modules is creating an
impossible provider version situation?"

For configurations using many modules this sort of question is likely to
come up a lot once we support versioned providers.

As a bonus use-case, this command also shows explicitly when a provider
configuration is being inherited from a parent module, to help users to
understand where the configuration is coming from for each module when
some child modules provide their own provider configurations.
2017-06-09 14:03:59 -07:00
Martin Atkins 4ab8973520 core: provide config to all import context tests
We're going to use config to determine provider dependencies, so we need
to always provide a config when instantiating a context or we'll end up
loading no providers at all.

We previously had a test for running "terraform import -config=''" to
disable the config entirely, but this test is now removed because it makes
no sense. The actual functionality its testing still remains for now,
but it will be removed in a subsequent commit when we start requiring that
a resource to be imported must already exist in configuration.
2017-06-09 14:03:59 -07:00
Martin Atkins c835ef8ff3 Update tests for the new ProviderResolver interface
Rather than providing an already-resolved map of plugins to core, we now
provide a "provider resolver" which knows how to resolve a set of provider
dependencies, to be determined later, and produce that map.

This requires the context to be instantiated in a different way, so this
very noisy diff is a mostly-mechanical update of all of the existing
places where contexts get created for testing, using some adapted versions
of the pre-existing utilities for passing in mock providers.
2017-06-09 14:03:59 -07:00
Martin Atkins 7ca592ac06 core: use ResourceProviderResolver to resolve providers
Previously the set of providers was fixed early on in the command package
processing. In order to be version-aware we need to defer this work until
later, so this interface exists so we can hold on to the possibly-many
versions of plugins we have available and then later, once we've finished
determining the provider dependencies, select the appropriate version of
each provider to produce the final set of providers to use.

This commit establishes the use of this new mechanism, and thus populates
the provider factory map with only the providers that result from the
dependency resolution process.

This disables support for internal provider plugins, though the
mechanisms for building and launching these are still here vestigially,
to be cleaned up in a subsequent commit.

This also adds a new awkward quirk to the "terraform import" workflow
where one can't import a resource from a provider that isn't already
mentioned (implicitly or explicitly) in config. We will do some UX work
in subsequent commits to make this behavior better.

This breaks many tests due to the change in interface, but to keep this
particular diff reasonably easy to read the test fixes are split into
a separate commit.
2017-06-09 14:03:59 -07:00