Commit Graph

1347 Commits

Author SHA1 Message Date
James Bardin c017149b31 don't store prepared backend config
The backend gets to "prepare" the configuration before Configure is
called, in order to validate the values and insert defaults. We don't
want to store this value in the "config state", because it will often
not match the raw config after it is prepared, forcing unecessary
backend migrations during init.

Since PrepareConfig is always called before Configure, we can store the
config value directly, and assume that it will be prepared in the same
manner each time.
2019-05-24 14:51:18 -04:00
James Bardin ee9a618369 don't migrate backend during init without override
If the backend config hashes match during init, and there are no new
backend override options, then we assume the existing config is OK.
Since init should be idempotent, we should be able to run init with no
options or config changes, and not effect the backends at all.
2019-05-24 11:31:04 -04:00
James Bardin cefc927e48 failing test for backend re-init 2019-05-23 18:21:52 -04:00
Sander van Harmelen 8fd3aa91a0
Merge pull request #21234 from hashicorp/svh/f-init-v0.12
Always try to select a workspace after initialization (v0.12)
2019-05-23 14:34:31 +02:00
Radek Simko 12f7ac1374
command/format: Reduce extra whitespaces & new lines 2019-05-16 14:52:06 +01:00
Martin Atkins 63fa1ac418 vendor: go get github.com/hashicorp/hcl2@master
This includes a small fix to ensure the parser doesn't produce an invalid
body for block parsing syntax errors, and instead produces an incomplete
result that calling applications like Terraform can still analyze.

The problem here was affecting our version-constraint-sniffing code, which
intentionally tried to find a core version constraint even if there's a
syntax error so that it can report that a new version of Terraform is a
likely cause of the syntax error. It was working in most cases, unless
it was the "terraform" block itself that contained the error, because then
we'd try to analyze a broken hcl.Block with a nil body.

This includes a new test for "terraform init" that exercises this
recovery codepath.
2019-05-14 15:37:46 -07:00
Sander van Harmelen d9ba0c69ea Always try to select a workspace after initialization
There are a number of use cases that can require a user to select a workspace after initializing Terraform.

To make sure we cover all these use cases, we will always call the selectWorkspace method to verify a valid workspace is already selected or (if needed) offer to select one before moving on.
2019-05-07 21:46:45 +02:00
Martin Atkins 6adcc7ab73 vendor: go get github.com/zclconf/go-cty@master
cty now guarantees that sets of primitive values will iterate in a
reasonable order. Previously it was the caller's responsibility to deal
with that, but we invariably neglected to do so, causing inconsistent
ordering. Since cty prioritizes consistent behavior over performance, it
now imposes its own sort on set elements as part of iterating over them so
that calling applications don't have to worry so much about it.

This change also causes cty to consistently push unknown and null values
in sets to the end of iteration, where before that was undefined. This
means that our diff output will now consistently list additions before
removals when showing sets, rather than the ordering being undefined as
before.

The ordering of known, non-null, non-primitive values is still not
contractually fixed but remains consistent for a particular version of
cty.
2019-04-30 15:49:28 -07:00
Martin Atkins eed605ac05 [WIP] Re-enable the end-to-end tests (#20044)
* internal/initwd: Allow deprecated relative module paths

In Terraform 0.11 we deprecated this form but didn't have any explicit
warning for it. Now we'll still accept it but generate a warning. In a
future major release we will drop this form altogether, since it is
ambiguous with registry module source addresses.

This codepath is covered by the command/e2etest suite.

* e2e: Skip copying .exists file, if present

We use this only in the "empty" test fixture in order to let git know that
the directory exists. We need to skip copying it so that we can test
"terraform init -from-module=...", which expects to find an empty
directory.

* command/e2etests: Re-enable and fix up the e2etest "acctests"

We disabled all of the tests that accessed remote services like the
Terraform Registry while they were being updated to support the new
protocols we now expect. With those services now in place, we can
re-enable these tests.

Some details of exactly what output we print, etc, have intentionally
changed since these tests were last updated.

* e2e: refactor for modern states and plans

* command/e2etest: re-enable e2etests and update for tf 0.12 compatibility
plugin/discovery: mkdirAll instead of mkdir when creating cache dir
2019-04-29 13:03:24 -04:00
Sander van Harmelen e41c2ecdce
Merge pull request #21139 from hashicorp/svh/b-stdin
Make sure UIInput keeps working after being canceled
2019-04-29 17:58:29 +02:00
Sander van Harmelen 9ab2e9d8b2 Make sure UIInput keeps working after being canceled
Once you start reading from stdin, that is a blocking call that will
never finish. So when a context is canceled causing the input method to
return, the read will remain blocking in the running goroutine.

There isn't a real solution for it (e.g. its not possible to unblock the
read) so the only solution is to make the reader reusable.
2019-04-29 15:15:26 +02:00
James Bardin f79a768a4e command/format: take noop changes from lcs
When rendering the diff, the NoOp changes should come from the LCS
sequence, rather than the new sequence. The two indexes will not align
in many cases, adding the wrong new object or indexing out of bounds.
2019-04-27 11:28:02 -04:00
Przemysław Dąbek 9605b093d9 command/graph: use user-supplied plugin path when running graph command (#18083) 2019-04-17 13:48:11 -04:00
Kristin Laemmert c1079b59bd
command/state_list.go: fix bug loading user-defined state (#21015)
* command/state_list.go: fix bug loading user-defined state

If the user supplied a state path via the `-state` flag and terraform
was running in a workspace other than `default`, the state was not being
loaded properly. Fixes #19920
2019-04-15 12:22:07 -04:00
Kristin Laemmert 1c95b21c6c
command/output: get the state path from the workspace or CLI argument (#20994)
Previously this command was setting the state path to the default state
file, instead of honoring the backend configuration.
2019-04-12 07:37:27 -04:00
Martin Atkins 88e76fa9ef configs/configschema: Introduce the NestingGroup mode for blocks
In study of existing providers we've found a pattern we werent previously
accounting for of using a nested block type to represent a group of
arguments that relate to a particular feature that is always enabled but
where it improves configuration readability to group all of its settings
together in a nested block.

The existing NestingSingle was not a good fit for this because it is
designed under the assumption that the presence or absence of the block
has some significance in enabling or disabling the relevant feature, and
so for these always-active cases we'd generate a misleading plan where
the settings for the feature appear totally absent, rather than showing
the default values that will be selected.

NestingGroup is, therefore, a slight variation of NestingSingle where
presence vs. absence of the block is not distinguishable (it's never null)
and instead its contents are treated as unset when the block is absent.
This then in turn causes any default values associated with the nested
arguments to be honored and displayed in the plan whenever the block is
not explicitly configured.

The current SDK cannot activate this mode, but that's okay because its
"legacy type system" opt-out flag allows it to force a block to be
processed in this way anyway. We're adding this now so that we can
introduce the feature in a future SDK without causing a breaking change
to the protocol, since the set of possible block nesting modes is not
extensible.
2019-04-10 14:53:52 -07:00
Pam Selle 346e341ff2 Only display status link if public registry used
Updates to throw a specific error if using the defined public
registry vs. another registry.
2019-04-05 16:49:27 -04:00
Kristin Laemmert f8a5e17d3d
command/format: improve "source" of error messages regarding missing arguments (#20907)
* vendor: update hcl2 dependency
* command/format: revert diagnostic format behavior if snippet or highlight range is empty
2019-04-03 14:04:59 -04:00
Frederic 0f1f504c22 command/format: indicate in diff when adding an attribute forces replacement 2019-03-29 14:52:13 -07:00
Pam Selle ff7245f27c Add status link to make message more helpful 2019-03-27 16:22:37 -04:00
Pam Selle d72456d188 Add friendly error for when registry unresponsive
If the registry is unresponsive, you will now get an error
specific to this, rather than a misleading "provider unavailable" type
error. Also adds debug logging for when errors like this may occur
2019-03-27 14:39:14 -04:00
Kristin Laemmert d5bb09b13b
command/show: refresh is required to show remote state (#20818) 2019-03-25 16:28:35 -04:00
Justin Downing 1e32ae243c grammatical updates to comments and docs (#20195) 2019-03-21 14:05:41 -07:00
findkim 161fe47b34 plugin/discovery: improve providery discovery verification errors 2019-03-21 14:38:51 -05:00
Martin Atkins e918fa83ec command/format: Don't panic when item removed from list of objects
Due to these tests happening in the wrong order, removing an object from
the end of a sequence of objects would previously cause a bounds-check
panic.

Rather than a more severe rework of the logic here, for now we'll just
introduce an extra precondition to prevent the panic. The code that
follows already handles the case where there _is_ no new object (i.e. the
"old" object is being deleted) as long as we're able to pass through this
type-checking logic.

The new "JSON list of objects - removing item" test covers this problem
by rendering a diff for an object being removed from the end of a list
of objects within a JSON value.
2019-03-19 15:46:40 -07:00
Justin Campbell e696e063dc
Merge pull request #20674 from hashicorp/registry-warnings
plugin/discovery: Parse warnings from TF Registry
2019-03-18 13:12:26 -04:00
Justin Campbell e6316c9de6 plugin/discovery: Parse warnings from TF Registry
Terraform Registry (and other registry implementations) can now return
an array of warnings with the versions response. These warnings are now
displayed to the user during a `terraform init`.
2019-03-18 12:21:27 -04:00
Justin Campbell 24e13d8ec1 plugin/discovery: Return tfdiags from Get
Allows us to surface warnings to the user using the tfdiags interfaces.
2019-03-18 12:21:27 -04:00
Martin Atkins c39905e1a8 command: Fix various issues in the "terraform state ..." subcommands
In earlier refactoring we updated these commands to support the new
address and state types, but attempted to partially retain the old-style
"StateFilter" abstraction that originally lived in the Terraform package,
even though that was no longer being used for any other functionality.

Unfortunately the adaptation of the existing filtering to the new types
wasn't exact and so these commands ended up having a few bugs that were
not covered by the existing tests.

Since the old StateFilter behavior was the source of various misbehavior
anyway, here it's removed altogether and replaced with some simpler
functions in the state_meta.go file that are tailored to the use-cases of
these sub-commands.

As well as just generally behaving more consistently with the other
parts of Terraform that use the new resource address types, this commit
fixes the following bugs:

- A resource address of aws_instance.foo would previously match an
  resource of that type and name in any module, which disagreed with the
  expected interpretation elsewhere of meaning a single resource in the
  root module.

- The "terraform state mv" command was not supporting moves from a single
  resource address to an indexed address and vice-versa, because the old
  logic didn't need to make that distinction while they are two separate
  address types in the new logic. Now we allow resources that do not have
  count/for_each to be treated as if they are instances for the purposes
  of this command, which is a better match for likely user intent and for
  the old behavior.

Finally, we also clean up a little some of the usage output from these
commands, which hasn't been updated for some time and so had both some
stale information and some inaccurate terminology.
2019-03-18 09:19:55 -07:00
Kristin Laemmert c73a5fc07d
command/show: add provider name to resource changes (#20711) 2019-03-15 13:20:43 -07:00
Kristin Laemmert 9d0d564ec7
`terraform show` and `terraform providers schema -json` should return valid json (#20697)
* command/providers schemas: return empty json object if config parses successfully but no providers found
* command/show (state): return an empty object if state is nil
2019-03-14 14:52:07 -07:00
James Bardin 0569e39788 don't try to treat "null" as json in diff output
Trying to decode and write "null" as json will panic, since it decodes
to nil.
2019-03-14 17:20:42 -04:00
Kristin Laemmert a15a4acf2f
configs/configupgrade: detect possible relative module sources (#20646)
* configs/configupgrade: detect possible relative module sources

If a module source appears to be a relative local path but does not have
a preceding ./, print a #TODO message for the user.

* internal/initwd: limit go-getter detectors to those supported by terraform
* internal/initwd: move isMaybeRelativeLocalPath check into getWithGoGetter

To avoid making two calls to getter.Detect, which potentially makes
non-trivial API calls, the "isMaybeRelativeLocalPath" check was moved to
a later step and a custom error type was added so user-friendly
diagnostics could be displayed in the event that a possible relative local
path was detected.
2019-03-13 11:17:14 -07:00
James Bardin a8df5586c0 check for errors when loading a backend config
We can't dereference the config module if there were any errors loading
it.
2019-03-12 17:57:26 -04:00
Martin Atkins 2d41c1009b command/format: Diffs for NestingMap block types
Our initial prototype of new-style diff rendering excluded this because
the old SDK has no support for this construct. However, we want to be able
to introduce this construct in the new SDK without breaking compatibility
with existing versions of Terraform Core, so we need to implement it now
so it's ready to be used once the SDK implements it.

The key associated with each block allows us to properly correlate the
items to recognize the difference between an in-place update of an
existing block and the addition/deletion of a block.
2019-03-11 08:18:26 -07:00
Martin Atkins dd1fa322a7 command/format: Support list/map blocks with dynamic-typed attrs
Our null-to-empty normalization was previously assuming these would always
be collection types, but that isn't true when a block contains something
dynamic since we must then use tuple or object types instead to properly
represent all of the individual element types.
2019-03-11 08:18:26 -07:00
Martin Atkins 69772b11b1 command/format: test for diff rendering with dynamic-typed subattrs
We use cty a little differently when a nested list block contains a
dynamically-typed attribute: it appears as a tuple value instead of a
list value so that we can retain the individual types of each element.

Here we introduce a test for that case, but doing so required also making
the runTestCases function handle types in a stricter way so that it will
produce planned values that match how Terraform Core would do it,
including the necessary late-bound type information for the
dynamically-typed attribute.
2019-03-11 08:18:26 -07:00
Kristin Laemmert 30672faebe
remove indentation from json output of `show` and `providers schema` (#20623)
* command/providers schema: correct help text

* command/show: remove indentation from json output

* command/providers schema: remove indentation from json output
2019-03-11 07:29:36 -07:00
James Bardin b8e53255b4 Revert "remove NormalizeObjectFromLegacySDK from diff"
This reverts commit 5c40d6610c.
2019-03-08 17:32:41 -05:00
Sander van Harmelen 973e2a7cf9 core: add a context to the UIInput interface 2019-03-08 10:24:40 +01:00
James Bardin a95d97f066
Merge pull request #20595 from hashicorp/jbardin/normalize-objects
normalize all objects read from the provider
2019-03-06 17:13:23 -05:00
James Bardin 5c40d6610c remove NormalizeObjectFromLegacySDK from diff
The NormalizeObjectFromLegacySDK calls have been moved into the
provider shims, so all objects generated by the provider should conform
now.
2019-03-06 16:21:32 -05:00
Brian Flad 20a814028d
command: Consistency implement and document parallelism default of 10
References:
* f4da82a023/command/command.go (L41)
* b9d8e96e0c/terraform/context.go (L149-L155)
2019-03-06 09:25:36 -05:00
Kristin Laemmert b9d8e96e0c
command/plan: plan output should indicate if a resource is being (#20580)
replaced because the instance was tainted.
2019-03-05 16:18:55 -08:00
Kristin Laemmert d72defd044
command/show and state show: honor user-specified plugin-dir (#20557)
Previously, these commands were not checking if the user specified a
`-plugin-dir` flag during `terraform init` and would therefor fail if
providers were not in one of the standard directories.

Fixes #20547
2019-03-05 08:32:11 -08:00
James Bardin 369d512e22
Merge pull request #20546 from hashicorp/jbardin/backend-init-panic
don't panic of the users aborts backend input
2019-03-01 19:03:49 -05:00
James Bardin 2adf5801d9 don't panic of the users aborts backend input
When the user aborts input, it may end up as an unknown value, which
needs to be converted to null for PrepareConfig.

Allow PrepareConfig to accept null config values in order to fill in
missing defaults.
2019-03-01 18:45:06 -05:00
Kristin Laemmert 8fb4e5ce6e
command/show: differentiate between state schemas and plan schemas. (#20516)
When a planfile is supplied to the `terraform show -json` command, the
context that loads only included schemas for resources in the plan. We
found an edge case where removing a data source from the configuration
(though only if there are no managed resources from the same provider)
would cause jsonstate.Marshal to fail because the provider schema wasn't
in the plan context.

jsonplan.Marshal now takes two schemas, one for plan and one for state.
If the state schema is nil it will simply use the plan schemas.
2019-03-01 13:59:57 -08:00
Kristin Laemmert c4151b7c7c
command/show: fixing bugs in modulecalls (#20513)
* command/show: fixing bugs in modulecalls

jsonconfig and jsonplan both had subtle bugs with the logic for
marshaling module calls that only showed up when multiple modules were
referenced. This PR fixes those bugs and extends the existing tests to
include multiple modules.

* sort all the things, mostly for tests
2019-03-01 13:59:12 -08:00
Kristin Laemmert 21f6e3dffd
Mildwonkey/012 docs updated (#20542)
* docs: update plan command documentation. Fixes #19235

* docs: added a missing reserved variable name. Fixes #19159.

* website: add note that resource names cannot start with a number

* website: add some notes to the 0.12 upgrade guide
2019-03-01 12:30:51 -08:00
Martin Atkins f193b11073 command/format: Normalize before/after values before rendering
We are now allowing the legacy SDK to opt out of the safety checks we try
to do after plan and apply, and so in such cases the before/after values
in planned changes may be inconsistent with our usual rules.

To avoid adding lots of extra complexity to the diff renderer to deal with
these situations, instead we'll normalize the handling of nested blocks
prior to using these values.

In the long run it'd be better to do this normalization at the source,
immediately after we receive an object from a provider using the opt-out,
but we're doing this at the outermost layer for now to avoid risking
unintended impacts on other Terraform Core components when we're just
about to enter the beta phase of the v0.12.0 release cycle.
2019-02-27 16:53:29 -08:00
James Bardin f43b87a61e
Merge pull request #20480 from hashicorp/jbardin/backend-init
backend init defaults and inputs
2019-02-26 15:04:00 -05:00
James Bardin c814f2da37 Change backend.ValidateConfig to PrepareConfig
This mirrors the change made for providers, so that default values can
be inserted into the config by the backend implementation. This is only
the interface and method name changes, it does not yet add any default
values.
2019-02-25 18:37:20 -05:00
Martin Atkins 6b6be3af35 command: Remove promise of plan -validate-only from validate docs
We brought forward a new implementation of "terraform validate" that was
originally scheduled for a later release after finding that it would be
simpler than reworking the old implementation for new v0.12 assumptions,
but we didn't yet implement "terraform plan -validate-only" in spite of
it being mentioned in the updated docs for "terraform validate".

For now then, the documentation will make the weaker suggestion of running
"terraform plan" to validate a particular _run_ rather than a particular
_module_, which is the closest thing we have for now. At some point after
v0.12.0 we will evaluate whether a validate-only mode for "terraform plan"
(which could then run without configuring the providers at all) is needed.
2019-02-25 14:27:59 -08:00
Kristin Laemmert 16823f43de
command/jsonprovider: export providers schemas to json (#20446)
* command/jsonprovider: a new package for exporting providers schemas as JSON
2019-02-25 13:32:47 -08:00
Martin Atkins 552dddfb4c command: Specialized error message for var decls in tfvars
A common new-user mistake is to place variable _declarations_ into .tfvars
files instead of variable _values_. To guide towards the correct approach
here, we add a specialized error message for that situation that includes
guidance on the distinction between declaring and setting values for
variables, and an example of what setting a value should look like.
2019-02-22 18:04:01 -08:00
Kristin Laemmert 4dd8192ec7
Merge pull request #20352 from hashicorp/mildwonkey/show-docs
mildwonkey/show-docs
2019-02-22 08:19:01 -08:00
Nick Fagerlund 3c42508700
Apply suggestions from code review
Co-Authored-By: mildwonkey <mildwonkey@users.noreply.github.com>
2019-02-21 11:52:08 -08:00
Kristin Laemmert 0c94e20a83
command/show enhancements and bugfixes
* command/jsonconfig: provider config marshaling enhancements

This PR fixes a bug wherein the keys in "provider_config" were the
"addrs.ProviderConfig", and therefore being overwritten for each module,
instead of the intended "addrs.AbsProviderConfig".

We realized that there was still opportunity for ambiguity, for example
if a user made a provider alias that was the same name as a module, so
we opted to use the syntax `modulename:providername(.provideralias)`

* command/json*: fixed a bug where we were attempting to lookup schemas
with the provider name, instead of provider type.
2019-02-20 14:27:49 -08:00
Kristin Laemmert b14472f22c
command/jsonconfig: add missing fields from configuration output (#20387)
Display depends_on for resources and outputs, and description for
outputs.
2019-02-19 16:31:10 -08:00
Kristin Laemmert 874b333962
command/format: fix an issue where data resources were not displaying (#20386)
Fixes #20245
2019-02-19 14:18:47 -08:00
Kristin Laemmert c59a274e96
command/jsonplan: "deposed" should be a string, not a bool. (#20351) 2019-02-19 13:55:49 -08:00
Kristin Laemmert 818b4ec068
command/show: add "module_version" to "module_calls" in config (#20367)
* command/show: add "module_version" to "module_calls" in config portion
of `terraform show`.

Also extended the `terraform show -json` test to run `init` so we could
add examples with modules. This does _not_ test the "module_version"
yet, but it _did_ help expose a bug in jsonplan where modules were
duplicated. This is also fixed in this PR.

* command/jsonconfig: rename version to version_constraint and
resolved_source to source.
2019-02-19 08:12:33 -08:00
Kristin Laemmert a43004e382 website/docs: add detailed documentation for the json serialization of
terraform plan
2019-02-14 14:20:54 -08:00
Kristin Laemmert f7ab90207c website/docs: add -json flag to terraform show docsn 2019-02-14 14:20:54 -08:00
Kristin Laemmert f783ed0d45
command/jsonconfig: display module variables in config output (#20311)
* command/jsonconfig: display module variables in config output

The tests have been updated to reflect this change.

* command/jsonconfig: properly handle variables with nil defaults
2019-02-12 12:03:07 -08:00
Martin Atkins eb1346447f
Merge #20282: Enforce expected behaviors for provider PlanResourceChange
An exception remains for the legacy SDK, which does not meet all of these requirements.
2019-02-12 09:19:05 -08:00
Martin Atkins 0b7179c363 command: Apply tests with realistic mock providers
Now that we're actually verifying correct behavior of providers during
plan and apply, our mock providers need to behave like real providers,
properly propagating any configured values through the plan and into the
final state.

For most of these it was simpler to just switch over to using the newer
PlanResourceChangeFn mock interface, away from the legacy DiffFn approach,
because then we can just return the ProposedNewState verbatim because our
schema for these tests does not require any default values to be
populated.
2019-02-11 17:46:55 -08:00
Kristin Laemmert f2f35265bc
command/show: json output enhancements (#20291)
* command/jsonplan: 
- add variables to plan output
- print known planned values for resources

Previously, resource attribute values were only displayed if the values
were wholly known. Now we will filter the unknown values out of the
change and print the known values.

* command/jsonstate: added depends_on and tainted fields
* command/show: update tests to reflect added fields
2019-02-11 13:17:03 -08:00
Martin Atkins 7e186f72d9 command: Update "terraform show -json" tests for changed provider contract
We now require a provider to populate all of its defaults -- including
unknown value placeholders -- during PlanResourceChange. That means the
mock provider for testing "terraform show -json" must now manage the
population of the computed "id" attribute during plan.

To make this logic a little easier, we also change the ApplyResourceChange
implementation to fill in a non-null id, since that makes it easier for
the mock PlanResourceChange to recognize when it needs to populate that
default value during an update.
2019-02-08 11:58:21 -08:00
Kristin Laemmert c810e4582c
command/show: continued work on `terraform show -json` output (#20171)
* command/jsonstate: do not hide SchemaVersion of '0'
* command/jsonconfig: module_calls should be a map
* command/jsonplan: include current terraform version in output
* command/jsonconfig: properly marshal expressions from a module call

Previously this was looking at the root module's variables, instead of
the child module variables, to build the module schema. This fixes that
bug.
2019-02-01 13:47:18 -08:00
Radek Simko 101454a609
command: Fix TestUiHookPostApply_emptyState 2019-01-31 12:25:57 +00:00
Kristin Laemmert 653bb74403
command/format: include nested blocks in `terraform show` output (#20149)
* command/format: include nested blocks in terraform show output
* command/format: fix tests
2019-01-30 10:08:59 -08:00
Radek Simko 37f397dded
command/test: Clarify failure (add full path) 2019-01-29 22:16:41 +00:00
Kristin Laemmert a2ac491cde
command/show: improvements to json output (#20139)
* command/show: add support for -json output for state

* command/jsonconfig: do not marshal empty count/for each expressions

* command/jsonstate: continue gracefully if the terraform version is somehow missing from state
2019-01-28 15:53:53 -08:00
Kristin Laemmert 6e057c529e
command/jsonplan: sort resources by address (#20113)
* command/jsonplan: sort resources by address
* command/show: extend test case to include resources with count
* command/json*: document resource ordering as consistent but undefined
2019-01-25 09:17:40 -08:00
Kristin Laemmert 514ac6b890
command/show: improvements to show -json output (#20110)
* terraform_version is now included in state
* provisioner "name" is now provisioner "type"
2019-01-24 15:28:53 -08:00
Kristin Laemmert e9099b4fcc
command/jsonplan: fix panic when filteredAfter is null (#20096)
* command/jsonplan: fix panic when filteredAfter is null
* command/jsonconfig: provider short name is required to properly look up resource schema
2019-01-23 16:14:34 -08:00
Kristin Laemmert f00fcb90bf
mildwonkey/b-show-state (#20032)
* command/show: properly marshal attribute values to json

marshalAttributeValues in jsonstate and jsonplan packages was returning
a cty.Value, which json/encoding could not marshal. These functions now
convert those cty.Values into json.RawMessages.

* command/jsonplan: planned values should include resources that are not changing
* command/jsonplan: return a filtered list of proposed 'after' attributes

Previously, proposed 'after' attributes were not being shown if the
attributes were not WhollyKnown. jsonplan now iterates through all the
`after` attributes, omitting those which are not wholly known.

The same was roughly true for after_unknown, and that structure is now
correctly populated. In the future we may choose to filter the
after_unknown structure to _only_ display unknown attributes, instead of
all attributes.

* command/jsonconfig: use a unique key for providers so that aliased
providers don't get munged together

This now uses the same "provider" key from configs.Module, e.g.
`providername.provideralias`.

* command/jsonplan: unknownAsBool needs to iterate through objects that are not wholly known

* command/jsonplan: properly display actions as strings according to the RFC,
instead of a plans.Action string.

For example:
a plans.Action string DeleteThenCreate should be displayed as ["delete",
"create"]

Tests have been updated to reflect this.

* command/jsonplan: return "null" for unknown list items.

The length of a list could be meaningful on its own, so we will turn
unknowns into "null". The same is less likely true for maps and objects,
so we will continue to omit unknown values from those.
2019-01-23 11:46:53 -08:00
Radek Simko b492c3662c
Merge pull request #20089 from hashicorp/t-cmd-fmt-sensitive-update
command/format: Add test to cover update of sensitive field
2019-01-23 15:55:53 +00:00
Radek Simko f04d0b48bc
command/format: Add test to cover update of sensitive field 2019-01-23 15:32:13 +00:00
Radek Simko 953eae7e4b
command/format: Fix rendering of different types 2019-01-23 13:13:48 +00:00
Radek Simko f3d1565d6f
command/format: Fix tuple diff formatting 2019-01-23 11:17:55 +00:00
Radek Simko 0dff8fe5e0
Add failing test case for tuple 2019-01-22 16:49:49 +00:00
Radek Simko 3d0a25c65d
command/format: Fix nested (JSON) object formatting 2019-01-22 16:26:28 +00:00
Radek Simko c5ba7469be
command/format: Fix rendering of unknown elements in set/map/list 2019-01-21 15:05:59 +00:00
Radek Simko 98cc99e632
command/format: Add crashing test with unknown element in map 2019-01-21 15:04:13 +00:00
Radek Simko 73225c7aeb
command/format: Add crashing test with unknown element in list 2019-01-21 15:04:13 +00:00
Radek Simko e7e8b7358f
command/format: Add crashing test with unknown element in set 2019-01-21 15:04:13 +00:00
Martin Atkins 8b094f48f7 command: Update "terraform get" to use the new module installer
We missed this on the initial update pass because this was calling
directly into the module package API rather than going through the Meta
methods that we updated for the new config loader.

m.installModules here is the same method that "terraform init" is using
for this purpose, ensuring the two will behave the same way. This changes
the output a little compared to the old installer, but it still includes
the important information about where each module is coming from.
2019-01-17 16:52:34 -08:00
Martin Atkins 176a5abfd3 command: Restore single-file support in "terraform fmt"
This possibility was lost in the rewrite to use HCL2, but it's used by
a number of external utilities and text editor integrations, so we'll
restore it here.

Using the stdin/stdout mode is generally preferable for text editor use
since it allows formatting of the in-memory buffer rather than directly
the file on disk, but for editors that don't have support for that sort of
tooling it can be convenient to just launch a single command and directly
modify the on-disk file.
2019-01-17 14:21:18 -08:00
Martin Atkins b0a43cab84 command: "terraform fmt" must fail if file has invalid syntax
Since the HCL formatter only works with tokens, it can in principle be
called with any input and produce some output. However, when given invalid
syntax it will tend to produce nonsensical results that may drastically
change the input file and be hard for the user to undo.

Since there's no strong reason to try to format an invalid or incomplete
file, we'll instead try parsing first and fail if parsing does not
complete successfully.

Since we talk directly to the HCL API here this is only a _syntax_ check,
and so it can be applied to files that are invalid in other ways as far
as Terraform is concerned, such as using unsupported top-level block types,
resource types that don't exist, etc.
2019-01-17 14:21:18 -08:00
findkim 7816e61614 Bump installer protocol version to 5 and separate client and server protocol references 2019-01-16 15:07:57 -06:00
Radek Simko 13896d72c5
command/format: Render empty object as {} 2019-01-15 14:34:49 +00:00
Radek Simko 5f14b7a7f2
command/format: Render empty primitive list/set as [] 2019-01-15 14:34:49 +00:00
Radek Simko 09d19ca9d9
command/format: Render empty JSON object as {} 2019-01-15 14:34:48 +00:00
Radek Simko 0dd2d56f18
command/format: Render empty maps as {} 2019-01-15 14:34:46 +00:00
Kim Ngo 41a2376915
Merge pull request #19981 from findkim/colorize-protocol-mismatch-msg
Condense protocol mismatch UI error message
2019-01-14 15:12:09 -06:00
findkim 6e0de3e3f5 Nest incompatible provider protocol error to include dynamic custom msg 2019-01-14 14:56:47 -06:00
Radek Simko aae87c7921
Merge pull request #19990 from hashicorp/f-cmd-fmt-ignore-empty-str
command/format: Ignore removal of empty strings
2019-01-14 20:33:43 +00:00
Martin Atkins 86c02d5c35 command: "terraform init" can partially initialize for 0.12upgrade
There are a few constructs from 0.11 and prior that cause 0.12 parsing to
fail altogether, which previously created a chicken/egg problem because
we need to install the providers in order to run "terraform 0.12upgrade"
and thus fix the problem.

This changes "terraform init" to use the new "early configuration" loader
for module and provider installation. This is built on the more permissive
parser in the terraform-config-inspect package, and so it allows us to
read out the top-level blocks from the configuration while accepting
legacy HCL syntax.

In the long run this will let us do version compatibility detection before
attempting a "real" config load, giving us better error messages for any
future syntax additions, but in the short term the key thing is that it
allows us to install the dependencies even if the configuration isn't
fully valid.

Because backend init still requires full configuration, this introduces a
new mode of terraform init where it detects heuristically if it seems like
we need to do a configuration upgrade and does a partial init if so,
before finally directing the user to run "terraform 0.12upgrade" before
running any other commands.

The heuristic here is based on two assumptions:
- If the "early" loader finds no errors but the normal loader does, the
  configuration is likely to be valid for Terraform 0.11 but not 0.12.
- If there's already a version constraint in the configuration that
  excludes Terraform versions prior to v0.12 then the configuration is
  probably _already_ upgraded and so it's just a normal syntax error,
  even if the early loader didn't detect it.

Once the upgrade process is removed in 0.13.0 (users will be required to
go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can
be simplified to remove that special mode, but the idea of doing the
dependency version checks against the liberal parser will remain valuable
to increase our chances of reporting version-based incompatibilities
rather than syntax errors as we add new features in future.
2019-01-14 11:33:21 -08:00
Martin Atkins 0c0a437bcb Move module install functionality over to internal/initwd 2019-01-14 11:33:21 -08:00
Radek Simko d96f4fa77b
command/format: Ignore removal of empty strings 2019-01-13 22:56:04 +00:00
Kristin Laemmert 5df9cd0f52
command/show: tests for -json output (#19980)
* command/show: added test scaffold for json output

More test cases will be added once the basic shape of the tests is
validated.

- command/json* packages now sort resources by address, matching
behavior elsewhere
- using cmp in tests instead of reflect.DeepEqual for the diffs
- updating expected output in tests to match sorting
2019-01-11 15:13:55 -08:00
Radek Simko bc4b7cad68
command/format: Render null in dark gray (#19616) 2019-01-11 19:27:09 +00:00
Kristin Laemmert cdf7cc2449
command/json*: updating documentation and adding tests (#19944)
A few minor fixes and cleanups as a result of said tests. Hooray for
eventual consistency!
2019-01-09 08:59:11 -08:00
Sander van Harmelen fb57e9b26f command/state: add proper locking 2019-01-08 14:57:52 +01:00
Kristin Laemmert 3e9c51c726
command/jsonstate: marshal state into a machine readable blob o'json (#19911)
* command/jsonstate: marshal state into a machine readable blob o'json
* consistent json tags are nice
2019-01-03 12:08:03 -08:00
Kristin Laemmert 2b6dc13f29
jsonplan: remove "proposed_unknown" structure in favor of "after_unknown" field in "change" (#19709)
* jsonplan: remove "proposed_unknown" structure in favor of
"after_unknown" field in "change"
2018-12-20 14:30:18 -08:00
Martin Atkins 2cf63d068f command: Always normalize config path before operations
Previously we were doing this rather inconsistently: some commands would
do it and others would not. By doing it here we ensure we always apply the
same normalization, regardless of which operation we're running.

This normalization is mostly for cosmetic purposes in error messages, but
it also ends up being used to populate path.module and path.root and so
it's important that we always produce consistent results here so that
we don't produce flappy changes as users work with different commands.

The fact that thus mutates a data structure as a side-effect is not ideal
but this is the best place to ensure it always gets applied without doing
any significant refactoring, since everything after this point happens in
the backend package where the normalizePath method is not available.
2018-12-19 13:47:42 -08:00
Kristin Laemmert 126e5f337f
json output of terraform plan (#19687)
* command/show: adding functions to aid refactoring

The planfile -> statefile -> state logic path was getting hard to follow
with blurry human eyes. The getPlan... and getState... functions were
added to help streamline the logic flow. Continued refactoring may follow.

* command/show: use ctx.Config() instead of a config snapshot

As originally written, the jsonconfig marshaller was getting an error
when loading configs that included one or more modules. It's not clear
if that was an error in the function call or in the configloader itself,
  but as a simpler solution existed I did not dig too far.

* command/jsonplan: implement jsonplan.Marshal

Split the `config` portion into a discrete package to aid in naming
sanity (so we could have for example jsonconfig.Resource instead of
jsonplan.ConfigResource) and to enable marshaling the config on it's
own.
2018-12-19 11:08:25 -08:00
James Bardin 149ccd929e missing commits from 19688 2018-12-18 16:54:09 -08:00
James Bardin 9667e06a03 decode backend hash as uint64
Older versions of terraform could save the backend hash number in a
value larger than an int.

While we could conditionally decode the state into an intermediary data
structure for upgrade, or detect the specific decode error and modify
the json, it seems simpler to just decode into the most flexible value
for now, which is a uint64.
2018-12-18 17:57:44 -05:00
Charles Kenney 2dfc3a399d command: allow -no-color option on "providers" command 2018-12-17 10:55:16 -08:00
Radek Simko f64978b64c
backend/local: Render CBD replacement (+/-) correctly (#19642)
* backend/local: Render CBD replacement (+/-) correctly

* command/format: Use IsReplace helper function
2018-12-14 13:45:47 +00:00
Masayuki Morita 33e1839730 command/hook_ui: Truncate the ID considering multibyte characters
Fixes #18822

The `tuncatedId` function had been introduced in #12261 and increased the
`maxIdLen` to 80 in #13317. Since the number of bytes itself seems to be
unimportant, the ID should be truncated to 80 characters, not 80 bytes.
2018-12-13 11:18:14 +09:00
Radek Simko b28efa0bf3
command/format: Fix tests 2018-12-12 14:28:12 +00:00
Radek Simko 0b981fa641
command/format: Fix rendering of force-new updates 2018-12-12 14:19:17 +00:00
Radek Simko fe117e9f02
command/format: Fix rendering of nested blocks during update 2018-12-11 18:07:21 +00:00
Radek Simko 2df886397b
command/format: Add more tests to cover non-primitive fields 2018-12-11 14:16:25 +00:00
Radek Simko e3e459a8d4
command/format: Restructure tests 2018-12-11 14:16:17 +00:00
Radek Simko 49e7026bdd
command/format: Add tests for ResourceChange renderer 2018-12-10 17:42:45 +00:00
Kristin Laemmert e3cecb7d56
format/state: added missing newline in the `outputs` output (#19542) 2018-12-04 11:37:22 -08:00
Radek Simko 9e5677adeb
Merge pull request #19453 from hashicorp/b-whole-body-diag-rendering
command/format: Fix rendering of attribute-agnostic diagnostics
2018-11-26 23:52:21 +00:00
Radek Simko 501fcd1e21
Fix tests after upgrading hcl 2018-11-26 23:38:37 +00:00
Radek Simko 4f2868a019
command/format: Fix rendering of attribute-agnostic diagnostics 2018-11-26 23:38:37 +00:00
Sander van Harmelen ef9054562e commands: make sure the correct flagset is used
A lot of commands used `c.Meta.flagSet()` to create the initial flagset for the command, while quite a few of them didn’t actually use or support the flags that are then added.

So I updated a few commands to use `flag.NewFlagSet()` instead to only add the flags that are actually needed/supported.

Additionally this prevents a few commands from using locking while they actually don’t need locking (as locking is enabled as a default in `c.Meta.flagSet()`.
2018-11-23 16:13:34 +01:00
Sander van Harmelen 79a9a15879 command/state: lock when pushing state
Next to adding the locking for the `state push` command, this commit also fixes a small bug where the lock would not be propertly released when running the `state show` command.

And finally it renames some variables in the `[un]taint` code in order to try to standardize the var names of a few frequently used variables (e.g. statemgr.Full, states.State, states.SyncState).
2018-11-20 11:15:16 +01:00
Martin Atkins 884aa387b8 command: Use vendoring when building helper programs in tests
In a couple places in tests we execute a child "go build" to make a helper
program. Now that we're running in module mode, "go build" will normally
default to downloading and caching dependencies, which we don't want
because we're still using vendoring for the moment.

Therefore we need to instruct these child builds to use vendoring too,
avoiding the need to download all of the dependencies and ensuring that
we'll be building with the same dependencies that we'd use for a normal
build.
2018-11-19 11:41:52 -08:00
Martin Atkins 5255e85238 "go fmt" fixups
Apparently my editor is still not reliably formatting on save, so I missed
a few formatting quirks in these files.
2018-11-19 09:02:35 -08:00
Martin Atkins 73c9521a04 command/e2etest: Temporarily disable tests that access network
Several of these tests rely on external services (e.g. Terraform Registry)
that have not yet been updated to support the needs of Terraform v0.12.0,
so for now we'll skip all of these tests and wait until those systems have
been updated.

This should be removed before Terraform v0.12.0 final to enable these
tests to be used as part of pre-release smoke testing.
2018-11-19 09:02:35 -08:00
Martin Atkins 37bc187f95 command: "terraform output" mustn't panic when no state is present
This is verified by TestOutput_noArgs.
2018-11-19 09:02:35 -08:00
Martin Atkins 53b5b95ef5 command: Fix TestRefresh_backup
The local filesystem state manager no longer creates backup files eagerly,
instead creating them only if on first write there is already a snapshot
present in the target file.

Therefore for this test to exercise the codepaths it intends to we must
create an initial state snapshot for it to overwrite, creating the backup
in the process.

There are several other tests for this behavior elsewhere, so this test
is primarily to verify that the refresh command is configuring the backend
appropriately to get the backups written in the desired location.
2018-11-19 09:02:35 -08:00
Martin Atkins 762a173c7f command: Fix TestRefresh_outPath
We now only create a backup state file if the given output file already
exists, which it does not in this test.

(The behavior of creating the backup files is already covered by other
tests, so no need for this one go out of its way to do it.)
2018-11-19 09:02:35 -08:00
Martin Atkins e2ba90fdfa command: Fix TestMetaBackend_planLocalMatch
We now don't create a local state backup until the first snapshot write,
so we don't expect there to be a backup file until the end of the test.
(There is already a check at the end there, unmodified by this change.)
2018-11-19 09:02:35 -08:00
Martin Atkins 48601d261d states/statemgr: In Filesystem, back up output file, not input file
The filesystem backend has the option of using a different file for its
initial read.

Previously we were incorrectly writing the contents of that file out into
the backup file, rather than the prior contents of the output file. Now
we will always read the output file in RefreshState in order to decide
what we will back up but then we will optionally additionally read the
input file and prefer its content as the "current" state snapshot.

This is verified by command.TestMetaBackend_planLocalStatePath and
TestMetaBackend_configureNew, which are both now passing.
2018-11-19 09:02:35 -08:00
Martin Atkins 27abd9c6b8 command: Fix TestMetaBackend_localDoesNotDeleteLocal
The changes to how we handle setting the state path on the local backend
broke the heuristic we were using here for detecting migration from one
local backend to another with the same state path, which would by default
end up deleting the state altogether after migration.

We now use the StatePaths method to do this, which takes into account
both the default values and any settings that have been set.

Additionally this addresses a flaw in the old method which could
potentially have deleted all non-default workspace state files if the
"path" setting were changed without also changing the "workspace_dir"
setting. This new approach is conservative because it will preserve all
of the files if any one overlaps.
2018-11-19 09:02:35 -08:00
Martin Atkins be79bf0412 command: Fix TestPlan_outBackend
In an earlier change we fixed the "backendFromConfig" codepath to be
able to properly detect changes to the -backend-config arguments during
"terraform init", but this detection is too strict for the normal case
of running an operation in a previously-initialized directory.

Before any of the recent changes, the logic here was to selectively update
the hash to include -backend-config settings in the init case. Since
that late hash recalculation was confusing, here we take the alternative
path of using the hash only in the normal case and full value comparison
in the init case. Treating both of these cases separately makes things
marginally easier to follow here.
2018-11-19 09:02:35 -08:00
Martin Atkins b316e4ab56 command: Fix TestImport_remoteState
The import command was imposing the default state path at the CLI level,
rather than leaving that to be handled by the backend. As a result, the
output state was always forced to be terraform.tfstate, regardless of
the backend settings.
2018-11-19 09:02:35 -08:00
Martin Atkins 2b9f92be31 command: Partially fix TestMetaBackend_planLocalStatePath
This test is testing some strange implementation details of the old
local backend which do not hold with the new filesystem state manager.
Specifically, it was expecting state to be read from the stateOutPath
rather than the statePath, which makes no sense here because the backend
is configured to read from the default terraform.tfstate file (which does
not exist.)

There is another problem with this test which will be addressed in a
subsequent commit.
2018-11-19 09:02:35 -08:00
Martin Atkins aecb66d3db command: Fix TestMetaBackend_configuredChangeCopy_multiToNoDefaultWithoutDefault
As part of integrating the new "remote" backend we relaxed the requirement
that a "default" workspace must exist in all backends and now skip
migrating empty workspace states to avoid creating unnecessary "default"
workspaces when switching between backends that require it and backends
that don't, such as when switching from the local backend (which always
has a "default" workspace) to Terraform Enterprise.
2018-11-19 09:02:35 -08:00
Martin Atkins ec27526cc3 command: Fix TestMetaBackend_configuredChangeCopy_multiToMulti
This was failing because we now handle the settings for the local backend
a little differently as a result of decoding it with the HCL2 machinery.

Specifically, the backend.State* fields are now assumed to be what is
given in configuration, and any CLI overrides are maintained separately
in OverrideState* fields so that they can be imposed "just in time" in
StatePaths.

This is particularly important because OverrideStatePath (when set) is
used regardless of workspace name, while StatePath is a suitable value
only for the "default" workspace, with others needing to be constructed
from StateWorkspaceDir instead.
2018-11-19 09:02:35 -08:00
Martin Atkins 6c7cecfbd8 command: More logging during migration
This just finishes off the logging added in earlier commits to get all
the way through to the actual migration call.
2018-11-19 09:02:35 -08:00
Martin Atkins f39a5d0962 command: Fix various TestMetaBackend-prefix tests
Our new state model has a different implementation of "empty" that doesn't
consider lineage/serial, so we need to have some actual content in these
state fixtures to avoid them being skipped during state migrations.
2018-11-19 09:02:35 -08:00
Martin Atkins 22c84c71a4 command: Use statemgr.Import and statemgr.Export for state push and pull
We previously hacked around the import/export functionality being missing
in the statemgr layer after refactoring, but now it's been reintroduced
to fix functionality elsewhere we should use the centralized Import and
Export functions to ensure consistent behavior.

In particular, this pushes the logic for checking lineage and serial
during push down into the state manager itself, which is better because
all other details about lineage and serial are managed within the state
managers.
2018-11-19 09:02:35 -08:00
Martin Atkins aacbe1d14b command: Fix TestMetaBackend_configureNewWithState
This test was initially failing because its fixture had a state which our
new state models consider to be "empty", and thus it was not migrated.

After fixing that (by adding an output to the fixture), this revealed a
bug that the lineage was not being persisted through the migration. This
is fixed by using the statemgr.Migrate method instead of writing via the
normal Writer interface, which allows two cooperating state managers to
properly transfer the lineage and serial along with the state snapshot.
2018-11-19 09:02:35 -08:00
Martin Atkins 24046ab833 command: More TRACE logging for the Backend instantiation codepaths 2018-11-19 09:02:35 -08:00
Martin Atkins 2293391241 command: Fix TestMetaBackend_emptyWithExplicitState
This test was incorrectly updated in a previous iteration, with it
creating a modified state to write but then not actually writing it,
writing an empty test state instead.

This made the test fail because a backup state file is created only if
the new state snapshot is different to the old when written.
2018-11-19 09:02:35 -08:00
Martin Atkins 8c54da0ad2 command: TestInit_fromModule_explicitDest guard against other tests
Some other test is leaving behind a terraform.tfstate after it concludes,
which can cause this test to fail in a strange way due to picking up
extra provider requirements from that state.

This check doesn't fix that problem, but it at least makes the test fail
in a more helpful way to avoid time wasted trying to debug this test when
it's some other test that actually has the bug.
2018-11-12 15:19:55 -08:00
Martin Atkins ced06a4ca3 command: fix panic in TestStatePush_lineageMismatch
This test is currently failing due to the command completing successfully,
which would previously cause a panic because we didn't properly initialize
the MockUi and so its error buffer is nil unless written to.

(The failure this was masking will be fixed in a subsequent commit.)
2018-11-12 15:19:55 -08:00