Commit Graph

1708 Commits

Author SHA1 Message Date
Kristin Laemmert 8bab3dd374
command/state list: list resources in nested and expanded modules (#27268)
* command/state list: list resources in nested and expaneded modules

A few distinct bugs fixed in here:

There was a bug in the logic checking if a given module was the child of
the targetAddr, now fixed. That resolved the basic issue where resources
in nested submodules were not listed.

The logic around allowMissing needed some tweaking to allow for empty
modules, as long as those modules had submodules with resources. state
list is the only command using allowMissing with false so this felt safe
to do.

Finally I extended the logic so list would included expanded modules,
which is to say giving module.foo would result in resources from
module.foo[1], module.foo[0], etc.

* update state list docs to show that module filtering includes any nested
modules
2020-12-14 11:07:15 -05:00
Alisdair McDiarmid e7db580e67
Merge pull request #27265 from hashicorp/alisdair/validate-json-tests
command: Add tests for terraform validate -json
2020-12-11 13:36:36 -05:00
Kristin Laemmert 8a4891383c
console: normalize module path before building context (#27263)
Expressions such as "path.root" were returning the cwd (or modulePath),
instead of the usual _relative_ path. This commit normalizes the path
before building the context.
2020-12-11 13:22:06 -05:00
Alisdair McDiarmid f1b95788b9 command: Add tests for terraform validate -json
Also uncomment and fix some tests which had been skipped for a couple of
years. Those validate cases work now!

Note that these test cases and the JSON output are not especially
minimized, making them snapshot/golden tests. The output looks correct
at time of writing, and we don't expect to change validate significantly
any time soon, but if we do there will be some churn here.
2020-12-11 13:09:25 -05:00
Martin Atkins bab4979128 command/init: Remove the warnings about the "legacy" cache directory
We included these warnings in v0.14 after noticing that we'd accidentally
published some incorrect documentation about the purpose of the plugin
cache directory under .terraform/plugins. We switched to using
.terraform/providers instead so that we could treat any missing providers
that appear in the legacy directory as likely to be a result of following
that documentation, and thus produce this extra warning.

However, the further we get from v0.13 the more likely it is for this
warning to be a confusing false positive rather than something helpful,
and this is a non-trivial codepath requiring us to retain a concept that
we otherwise don't need (the "legacy cache dir"), so here we'll remove
those warnings and support code for v0.15 onwards.

These warnings were always accompanied by an error message saying that a
provider could not be found, and that error message remains after this
change. This just removes the "by the way..."-style warning we had been
emitting alongside the errors.
2020-12-11 08:00:15 -08:00
Martin Atkins 4b3e237668 command/init: Hint about providers in other namespaces
If a user forgets to specify the source address for a provider, Terraform
will assume they meant a provider in the registry.terraform.io/hashicorp/
namespace. If that ultimately doesn't exist, we'll now try to see if
there's some other provider source address recorded in the registry's
legacy provider lookup table, and suggest it if so.

The error message here is a terse one addressed primarily to folks who are
already somewhat familiar with provider source addresses and how to
specify them. Terraform v0.13 had a more elaborate version of this error
message which directed the user to try the v0.13 automatic upgrade tool,
but we no longer have that available in v0.14 and later so the user must
make the fix themselves.
2020-12-10 10:11:27 -08:00
Alisdair McDiarmid 4b9dada877 command: Revert local patch for pkg/browser
The upstream bug with opening a browser on Windows Subsystem for Linux
has been fixed, so this reverts our local patch for this. The approach
upstream adds fallback support for x-www-browser and www-browser if
xdg-open fails, and this fixes the problem on WSL.

This reverts commit 12e090ce48.
2020-12-10 09:27:02 -05:00
Martin Atkins 3268a7eaba command/output: Raw output mode
So far the output command has had a default output format intended for
human consumption and a JSON output format intended for machine
consumption.

However, until Terraform v0.14 the default output format for primitive
types happened to be _almost_ a raw string representation of the value,
and so users started using that as a more convenient way to access
primitive-typed output values from shell scripts, avoiding the need to
also use a tool like "jq" to decode the JSON.

Recognizing that primitive-typed output values are common and that
processing them with shell scripts is common, this commit introduces a new
-raw mode which is explicitly intended for that use-case, guaranteeing
that the result will always be the direct result of a string conversion
of the output value, or an error if no such conversion is possible.

Our policy elsewhere in Terraform is that we always use JSON for
machine-readable output. We adopted that policy because our other
machine-readable output has typically been complex data structures rather
than single primitive values. A special mode seems justified for output
values because it is common for root module output values to be just
strings, and so it's pragmatic to offer access to the raw value directly
rather than requiring a round-trip through JSON.
2020-12-09 10:10:02 -08:00
Pam Selle b963ea8594 Update docs and add warning for -get-plugins
As of Terraform 0.13+, the get-plugins command has been
superceded by new provider installation mechanisms, and
general philosophy (providers are always installed, but
the sources may be customized). Updat the init command
to give users a warning if they are setting this flag,
to encourage them to remove it from their workflow, and
update relevant docs and docstrings as well
2020-12-07 14:13:52 -05:00
James Bardin dcf0dba6f4
Merge pull request #27081 from hashicorp/jbardin/staticcheck
Fixes to pass static analysis
2020-12-02 15:43:10 -05:00
Kristin Laemmert 3fa063b8dc
command/format: concise diff is now the default (#27079)
* command/format: concise diff is no longer an experiment

Since state formatting goes through the "diff" printer, I have
repurposed the concise flag as a verbose flag, used only when printing
state. It's silly but it works!

* remove helper/experiment
With this experiment concluded, we no longer need helper/experiment. The
shadow experiment had not been touched in many years, so I removed all
references, and removed the package entirely. Any new experiments are
expected to be configuration experiments handled by our (other)
experiments package.

* check for the verbose flag consistently, in case we end up using it in plans in the future
2020-12-02 15:42:41 -05:00
James Bardin 0b4c96fa92 command/jsonprovider: staticcheck 2020-12-02 13:59:20 -05:00
James Bardin 41d4dd82d6 format staticcheck 2020-12-02 13:59:19 -05:00
James Bardin a1d41504f2 e2etest staticcheck 2020-12-02 13:59:19 -05:00
James Bardin 1c58c6ba48 command staticcheck 2020-12-02 13:59:19 -05:00
James Bardin ccf0d8815f staticcheck 2020-12-02 13:59:19 -05:00
James Bardin 2016b6bd1c validate is no longer called from there 2020-12-02 13:59:19 -05:00
James Bardin 8b08ebc4b5 consistent recievers 2020-12-02 13:59:19 -05:00
James Bardin 6f577abc72 unused 2020-12-02 13:59:19 -05:00
James Bardin f521fcca97 cleanup error handling and some for loops 2020-12-02 13:59:19 -05:00
James Bardin f8ccbcd3bb Add e2e test for provisioner plugin
Ensure we can still execute a 3rd party provisioner binary, using the
legacy plugin discovery location of the configuration root directory.
2020-12-02 12:45:00 -05:00
James Bardin 76cb40005a add grpcwrap.Provisioner
Rename grpcwrap.New() to grpcwrap.Provider()
Add a grpcwrap function to create a test proivisioner plugin.
2020-12-02 12:45:00 -05:00
James Bardin 5e089c2c09 run built-in provisioners in-process
Use the new provisioner interfaces, and run the built-in provisioners
in-process.
2020-12-02 12:45:00 -05:00
James Bardin dc9ded8618 remove old version call site 2020-12-02 12:45:00 -05:00
James Bardin e4c72015a3 remove old test provider from e2e tests 2020-12-02 12:45:00 -05:00
James Bardin e998882824 remove legacy types from command package 2020-12-02 12:33:18 -05:00
James Bardin 811a3a81e9 add test provider bin for e2etest
One e2etest still requires the bin, so add that back temporarily.
2020-12-02 12:16:36 -05:00
James Bardin 75bbf0b62b udpate e2etest to use internal/legacy
The use of this provider will be factored out, but just change the
import for now.
2020-12-02 12:16:35 -05:00
Pam Selle c6ab9b1553
Merge pull request #26938 from hashicorp/pselle/remove-vendor-provisioners
Remove vendor provisioners
2020-12-02 11:48:40 -05:00
James Bardin a75dcd4be0 warning are now in stdout 2020-11-30 12:38:11 -05:00
Alisdair McDiarmid 42437482e5
Merge pull request #26947 from hashicorp/alisdair/backend-validate-remote-backend-terraform-version
backend: Validate remote backend Terraform version
2020-11-20 13:50:05 -05:00
Martin Atkins 0a596d2a12 command/version: Report the current platform
Along with all of the other information we previously reported in the
"terraform version" output, we'll now include the name of the current
platform as our provider mechanisms represent it.

This is addressing a long-standing minor annoyance where we often can't
tell from an incomplete bug report which platform Terraform was running
on, and incomplete bug reporters do tend to at least include the
"terraform version" output even if they don't also include the requested
full trace log.

However, what motivated doing it _now_ is that anyone building a provider
registry or mirror needs to have some awareness of these platform
identifiers which have been, until v0.13, mostly an implementation detail.
This additional information is a small thing we can do to help registry
builders find out what the platform identifier ought to be for each of
the platforms they aim to support, even if some of them are platforms
which the Go compiler allows but which HashiCorp doesn't officially
support.

The new information is on a line of its own in the output as a pragmatic
way to avoid breaking anyone who might be using something like
$(terraform version | head -n1) to print a brief Terraform version
identifier into some logs. That's not an interface we officially support
for machine consumption, but it's easy to avoid breaking it here and so we
won't do so.
2020-11-19 14:15:30 -08:00
Alisdair McDiarmid c5c1f31db3 backend: Validate remote backend Terraform version
When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.

This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.

To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.

Terraform version compatibility is defined as:

- For all releases before 0.14.0, local must exactly equal remote, as
  two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
  version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
  we will not change the state version number in a patch release.

If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.

Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.

In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
2020-11-19 13:19:40 -05:00
Alisdair McDiarmid 21d80a26ea command: Fix fmt to preserve blank block labels 2020-11-18 11:59:10 -05:00
Pam Selle e39e0e3d04 Remove vendor provisioners and add fmt Make target
Remove chef, habitat, puppet, and salt-masterless provsioners,
which follows their deprecation. Update the documentatin for these
provisioners to clarify that they have been removed from later versions
of Terraform. Adds the fmt Make target back and updates fmtcheck script
for correctness.
2020-11-17 11:22:03 -05:00
Alisdair McDiarmid 10cc25fc21 terraform: Compare locks and provider requirements
When building a context, we read the dependency locks and ensure that
the provider requirements from the configuration can be satisfied.
If the configured requirements change such that the locks need to be
updated, we explain this and recommend running "terraform init".

This check is ignored for any providers which are locally marked as in
development. This includes unmanaged providers and those listed in the
provider installation `dev_overrides` block.
2020-11-06 12:58:52 -05:00
James Bardin e7b2d98ca3 Use prepared config in provider.Configure
Core is only using the PrepareProviderConfig call for the validation
part of the method, but we should be re-validating the final config
immediately before Configure.

This change elects to not start using the PreparedConfig here, since
there is no useful reason for it at this point, and it would
introduce a functional difference between terraform releases that can be
avoided.
2020-11-04 12:53:00 -05:00
Alisdair McDiarmid b90994deac
Merge pull request #26735 from hashicorp/alisdair/disable-terraform-state-file-version-check
states: Disable Terraform version check
2020-10-29 15:22:06 -04:00
Martin Atkins d24fdce5b8 command/format: Include unknown values in diagnostics
Previously when printing the relevant variables involved in a failed
expression evaluation we would just skip over unknown values entirely.

There are some errors, though, which are _caused by_ a value being
unknown, in which case it's helpful to show which of the inputs to that
expression were known vs. unknown so that the user can limit their further
investigation only to the unknown ones.

While here I also added a special case for sensitive values that overrides
all other display, because we don't know what about a value is sensitive
and so better to give nothing away at the expense of a slightly less
helpful error message.
2020-10-29 09:07:48 -07:00
James Bardin d1ac382ec9
Merge pull request #26738 from hashicorp/jbardin/eval-diagnostics
Eval diagnostics
2020-10-28 17:21:18 -04:00
Alisdair McDiarmid dc2e7520e5 states: Disable Terraform version check
For this version of Terraform and forward, we no longer refuse to read
compatible state files written by future versions of Terraform. This is
a commitment that any changes to the semantics or format of the state
file after this commit will require a new state file version 5.

The result of this is that users of this Terraform version will be able
to share remote state with users of future versions, and all users will
be able to read and write state. This will be true until the next major
state file version is required.

This does not affect users of previous versions of Terraform, which will
continue to refuse to read state written by later versions.
2020-10-28 16:52:35 -04:00
Pam Selle 92a8d2061c
Merge pull request #26739 from hashicorp/pselle/show-json
Unmark values before showing in JSON
2020-10-28 16:49:56 -04:00
Pam Selle f61a342311 Hide sensitive outputs in terraform show 2020-10-28 15:46:09 -04:00
Pam Selle 66091ae36c Unmark values before showing in JSON
This prevents "sensitive" values from unintentionally
showing as nil when running terraform show -json
2020-10-28 15:30:04 -04:00
James Bardin f987b69777 interrupted execution now exits with an error 2020-10-28 14:40:30 -04:00
Martin Atkins c94a6102df command: Improve consistency of the command short descriptions
The short description of our commands (as shown in the main help output
from "terraform") was previously very inconsistent, using different
tense/mood for different commands. Some of the commands were also using
some terminology choices inconsistent with how we currently talk about
the related ideas in our documentation.

Here I've tried to add some consistency by first rewriting them all in
the imperative mood (except the ones that just are just subcommand
groupings), and tweaking some of the terminology to hopefully gel better
with how we present similar ideas in our recently-updated docs.

While working on this I inevitably spotted some similar inconsistencies
in the longer-form help output of some of the commands. I've not reviewed
all of these for consistency, but I did update some where the wording
was either left inconsstent with the short form changes I'd made or
where the prose stood out to me as particularly inconsistent with our
current usual documentation language style.

All of this is subjective, so I expect we'll continue to tweak these over
time as we continue to develop our documentation writing style based on
user questions and feedback.
2020-10-26 09:55:21 -07:00
Martin Atkins 39504ede05 command: Remove the useless "debug" subcommand
This is just a husk of a container command that has no nested commands
under it, so it isn't serving any purpose.
2020-10-26 09:55:21 -07:00
James Bardin f8893785f0 separate core and provider loggers
Now that hclog can independently set levels on related loggers, we can
separate the log levels for different subsystems in terraform.

This adds the new environment variables, `TF_LOG_CORE` and
`TF_LOG_PROVIDER`, which each take the same set of log level arguments,
and only applies to logs from that subsystem. This means that setting
`TF_LOG_CORE=level` will not show logs from providers, and
`TF_LOG_PROVIDER=level` will not show logs from core. The behavior of
`TF_LOG` alone does not change.

While it is not necessarily needed since the default is to disable logs,
there is also a new level argument of `off`, which reflects the
associated level in hclog.
2020-10-23 12:46:32 -04:00
James Bardin bc1a841d65
Merge pull request #26665 from hashicorp/jbardin/logging
Restore "crash.log" behavior and remove prefixedio
2020-10-22 10:34:51 -04:00
James Bardin ef4fee5acb warnings are now on stderr 2020-10-21 18:24:09 -04:00