This commit extracts the remaining UI logic from the local backend,
and removes access to the direct CLI output. This is replaced with an
instance of a `views.Operation` interface, which codifies the current
requirements for the local backend to interact with the user.
The exception to this at present is interactivity: approving a plan
still depends on the `UIIn` field for the backend. This is out of scope
for this commit and can be revisited separately, at which time the
`UIOut` field can also be removed.
Changes in support of this:
- Some instances of direct error output have been replaced with
diagnostics, most notably in the emergency state backup handler. This
requires reformatting the error messages to allow the diagnostic
renderer to line-wrap them;
- The "in-automation" logic has moved out of the backend and into the
view implementation;
- The plan, apply, refresh, and import commands instantiate a view and
set it on the `backend.Operation` struct, as these are the only code
paths which call the `local.Operation()` method that requires it;
- The show command requires the plan rendering code which is now in the
views package, so there is a stub implementation of a `views.Show`
interface there.
Other refactoring work in support of migrating these commands to the
common views code structure will come in follow-up PRs, at which point
we will be able to remove the UI instances from the unit tests for those
commands.
* providers.Interface: huge renamification
This commit renames a handful of functions in the providers.Interface to
match changes made in protocol v6. The following commit implements this
change across the rest of the codebase; I put this in a separate commit
for ease of reviewing and will squash these together when merging.
One noteworthy detail: protocol v6 removes the config from the
ValidateProviderConfigResponse, since it's never been used. I chose to
leave that in place in the interface until we deprecate support for
protocol v5 entirely.
Note that none of these changes impact current providers using protocol
v5; the protocol is unchanged. Only the translation layer between the
proto and terraform have changed.
* command/format: check for sensitive NestedTypes
Eventually, the diff formatter will need to be updated to properly
handle NestedTypes, but for now we can let the existing function deal
with them as regular cty.Object-type attributes.
To avoid printing sensitive nested attributes, we will treat any
attribute with at least one sensitive nested attribute as an entirely
sensitive attribute.
* bugfix for Object ImpliedType()
ImpliedType() was returning too early when the given object had optional
attributes, therefore skipping the incredibly important step of
accounting for the nesting mode when returning said type.
Due to calling the Colorize function with the full string instead of the
format string, plan/apply logs which include resource instance keys or
IDs which happen to match color formatting would be rendered
incorrectly.
This commit fixes this by only colorizing the known-safe format string.
We also add full test coverage for the UI hook, although only one of the
hooks is tested for this color bugfix due to verbosity of the test.
We also add the bold coloring to the provisioner output prefix, which
seems to have been an oversight.
Now that the view code is separated, we can increase test coverage in
unit tests. This commit moves some tests from the command package which
were testing only view code, and adds more new test cases.
The clistate package includes a Locker interface which provides a simple
way for the local backend to lock and unlock state, while providing
feedback to the user if there is a delay while waiting for the lock.
Prior to this commit, the backend was responsible for initializing the
Locker, passing through direct access to the cli.Ui instance.
This structure prevented commands from implementing different
implementations of the state locker UI. In this commit, we:
- Move the responsibility of creating the appropriate Locker to the
source of the Operation;
- Add the ability to set the context for a Locker via a WithContext
method;
- Replace the Locker's cli.Ui and Colorize members with a StateLocker
view;
- Implement views.StateLocker for human-readable UI;
- Update the Locker interface to return detailed diagnostics instead of
errors, reducing its direct interactions with UI;
- Add a Timeout() method on Locker to allow the remote backend to
continue to misuse the -lock-timeout flag to cancel pending runs.
When an Operation is created, the StateLocker field must now be
populated with an implementation of Locker. For situations where locking
is disabled, this can be a no-op locker.
This change has no significant effect on the operation of Terraform,
with the exception of slightly different formatting of errors when state
locking or unlocking fails.
Move the code which renders Terraform hook callbacks as UI into the
views package, backed by a views.View instead of a cli.Ui. Update test
setup accordingly.
To allow commands to control this hook, we add a hooks member on the
backend Operation struct. This supersedes the hooks in the Terraform
context, which is not directly controlled by the command logic.
This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
The enhanced backends (local and remote) need to be able to render
diagnostics during operations. Prior to this commit, this functionality
was supported with a per-backend `ShowDiagnostics` function pointer.
In order to allow users of these backends to control how diagnostics are
rendered, this commit moves that function pointer to the `Operation`
type. This means that a diagnostic renderer is configured for each
operation, rather than once per backend initialization.
Some secondary consequences of this change:
- The `ReportResult` method on the backend is now moved to the
`Operation` type, as it needs to access the `ShowDiagnostics` callback
(and nothing else from the backend);
- Tests which assumed that diagnostics would be written to the backend's
`cli.Ui` instance are migrated to using a new record/playback diags
helper function;
- Apply, plan, and refresh commands now pass a pointer to the `Meta`
struct's `showDiagnostics` method.
This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
Rather than modifying and relying on the existing Meta.process
argument extractor, we can more clearly handle global CLI flags using
a separate parser step. This allows us to explicitly configure the view
in the command.
Terraform supports multiple output formats for several sub-commands.
The default format is user-readable text, but many sub-commands support
a `-json` flag to output a machine-readable format for the result. The
output command also supports a `-raw` flag for a simpler, scripting-
focused machine readable format.
This commit adds a "views" abstraction, intended to help ensure
consistency between the various output formats. This extracts the render
specific code from the command package, and moves it into a views
package. Each command is expected to create an interface for its view,
and one or more implementations of that interface.
By doing so, we separate the concerns of generating the sub-command
result from rendering the result in the specified output format. This
should make it easier to ensure that all output formats will be updated
together when changes occur in the result-generating phase.
There are some other consequences of this restructuring:
- Views now directly access the terminal streams, rather than the
now-redundant cli.Ui instance;
- With the reorganization of commands, parsing CLI arguments is now the
responsibility of a separate "arguments" package.
For now, views are added only for the output sub-command, as an example.
Because this command uses code which is shared with the apply and
refresh commands, those are also partially updated.
Errors encountered when parsing flags for apply, plan, and refresh were
being suppressed. This resulted in a generic usage error when using an
invalid `-target` flag.
This commit makes several changes to address this. First, these commands
now output the flag parse error before exiting, leaving at least some
hint about the error. You can verify this manually with something like:
terraform apply -invalid-flag
We also change how target attributes are parsed, moving the
responsibility from the flags instance to the command. This allows us to
customize the diagnostic output to be more user friendly. The
diagnostics now look like:
```shellsession
$ terraform apply -no-color -target=foo
Error: Invalid target "foo"
Resource specification must include a resource type and name.
```
Finally, we add test coverage for both parsing of target flags, and at
the command level for successful use of resource targeting. These tests
focus on the UI output (via the change summary and refresh logs), as the
functionality of targeting is covered by the context tests in the
terraform package.
The JSON plan output format includes a serialized, simplified version of
the configuration. One component of this config is a map of provider
configurations, which includes version constraints.
Until now, only version constraints specified in the provider config
blocks were exposed in the JSON plan output. This is a deprecated method
of specifying provider versions, and the recommended use of a
required_providers block resulted in the version constraints being
omitted.
This commit fixes this with two changes:
- When processing the provider configurations from a module, output the
fully-merged version constraints for the entire module, instead of any
constraints set in the provider configuration block itself;
- After all provider configurations are processed, iterate over the
required_providers entries to ensure that any configuration-less
providers are output to the JSON plan too.
No changes are necessary to the structure of the JSON plan output, so
this is effectively a semantic level bug fix.
The previous changes removing support for using the trailing positional
argument as a working directory missed a spot in the apply/destroy
command implementation. We still support this argument for applying a
saved plan:
terraform apply foo.tfplan
However, if you pass a positional path which doesn't "look like" a plan
(for example, the path to a configuration directory), Terraform would
silently ignore it and continue.
This commit fixes that by adding an error message if the user specifies
a path which the plan loader rejects as not "looking like" a plan. This
message includes a reference to the `-chdir` flag as a pointer about
what to do next.
We also rearrange the error message when calling `terraform destroy`
with a plan file argument, and add test coverage for the above. While
we're here, update the destroy tests to copy the fixture directory,
chdir, and defer cleanup.
This dramatically simplifies the logic around auto-approve, which is
nice.
Also add test coverage for the manual approve step, for both apply and
destroy, answering both yes and no.
To make the command arguments easier to understand and extend, we are
moving away from positional arguments. This commit changes the graph
command to take a `-plan` flag instead of an optional trailing path.
Several commands continued to support the legacy positional path
argument to specify a working directory. This functionality has been
replaced with the global -chdir flag, which is specified before any
other arguments, including the sub-command name.
This commit removes support for the trailing path parameter from
most commands. The only command which still supports a path argument is
fmt, which also supports "-" to indicate receiving configuration from
standard input.
Any invocation of a command with an invalid trailing path parameter will
result in a short error message, pointing at the -chdir alternative.
There are many test updates in this commit, almost all of which are
migrations from using positional arguments to specify a working
directory. Because of the layer at which these tests run, we are unable
to use the -chdir argument, so the churn in test files is larger than
ideal. Sorry!
CountHook is an implementation of terraform.Hook which is used to
calculate how many resources were added, changed, or destroyed during an
apply. This hook was previously injected in the local backend code,
which means that the apply command code has no access to these counts.
This commit moves the CountHook code into the command package, and
removes an unused instance of the hook in the plan code path. The goal
here is moving UI code into the command package.
The -module flag to terraform output has been unimplemented since 0.12.
This commit removes some dead code and the specific error message for
this flag.
The website documentation for output does not mention this flag, so it
is unchanged.
Commit e865faf adds visual indentation for diagnostic messages using various
vertical line characters. The present commit disables this behaviour when
running with colourised output disabled.
While the contents of stderr are not intended to be part of the Terraform API,
this is currently how the hashicorp/terraform-exec library detects certain
error types in order to present them as well-known Go errors to the user. Such
detection is complicated when vertical lines are added to the CLI output at
unpredictable points.
I expect this change will also be helpful for screen reader users.
Adds a test to make sure that this text doesn't accidentally
get tabs added, without having a test that simply direct matches
the whole strings (which would be brittle to adding a tab to the
test validation)
Previously the state migration process was using the fallback strict
error check when migrating to or from a Terraform Cloud workspace. This
resulted in an error when running init if the local and remote Terraform
versions did not exactly match.
This was excessively strict. When migrating from a remote Terraform
Cloud workspace to local state, there is no need for a version check at
all, as we cannot break the Terraform Cloud workspace. When migrating
to Terraform Cloud, we should use the more forgiving check, rather than
the strict equality.
This commit fixes both of these cases accordingly, and allows migrating
state to and from Terraform Cloud remote workspaces without errors.
I frequently see people attempting to ask questions about Terraform's
error and warning messages but either only copying part of the message or
accidentally copying a surrounding paragraph that isn't part of the
message.
While I'm sure some of these are just "careless" mistakes, I've also
noticed that this has sometimes overlapped with someone asking a question
whose answer is written directly in the part of the message they didn't
include when copying, and so I have a theory that our current output
doesn't create a good enough visual hierarchy for sighted users to
understand where the diagnostic messages start and end when we show them
in close proximity to other content, or to other diagnostic messages.
As a result, some folks fail to notice the relevant message that might've
answered their question.
I tried a few different experiments for different approaches here, such
as adding more horizontal rules to the output and coloring the detail
text differently, but the approach that felt like the nicest compromise
to me was what's implemented here, which is to add a vertical line
along the left edge of each diagnostic message, colored to match with the
typical color we use for each diagnostic severity. This means that the
diagnostics end up slightly indented from what's around them, and the
vertical line seems to help subtly signal how we intended the content
to be grouped together.
In some terminal emulators, writing a character into the last column on a
row causes the terminal to immediately wrap to the beginning of the next
line, even if the very next character in the stream is a hard newline.
That can then lead to errant blank lines in the final output which make
it harder to navigate the visual hierarchy.
As a compromise to avoid this, we'll format our horizontal rules and
paragraphs to one column less than the terminal width. That does mean that
our horizontal rules won't _quite_ cover the whole terminal width, but
it seems like a good compromise in order to get consistent behavior across
a wider variety of terminal implementations.
We were previously using some ASCII art to create some visual divisions
between parts of the diagnostic output. Now that we are requiring a UTF-8
terminal we can print out box drawing characters instead.
We now require the output to accept UTF-8 and we can determine how wide
the terminal (if any) is, so here we begin to make use of that for the
"terraform plan" command.
The horizontal rule is now made of box drawing characters instead of
hyphens and fills the whole terminal width.
The paragraphs of text in the output are now also wrapped to fill the
terminal width, instead of the hard-wrapping we did before.
This is just a start down the road of making better use of the terminal
capabilities. Lots of other commands could benefit from updates like these
too.
Here we propagate in the initialized terminal.Streams from package main,
and then onwards to backends running in CLI mode.
This also replaces our use of helper/wrappedstreams to determine whether
stdin is a terminal or a pipe. helper/wrappedstreams returns incorrect
file descriptors on Windows, causing StdinPiped to always return false on
that platform and thus causing one of the odd behaviors discussed in
Finally, this includes some wrappers around the ability to look up the
number of columns in the terminal in preparation for use elsewhere. These
wrappers deal with the fact that our unit tests typically won't populate
meta.Streams.
After verifying the remote backend workspace version is compatible with
the local Terraform version, we need to record that this check was
successful. Otherwise the fallback error handling path will run a strict
version check, even if the versions are compatible, which will cause an
unexpected failure.
The revision field is only populated on dev builds so this means
most releases of Terraform have an empty "terraform_revision" field
in the JSON output. Since we recommend developers use go tooling
to `go build` this tool when developing, the revision is not useful
data and so it is removed.
When running state mv with a resource source, but the destination
fails, provide a hint that the source is a resource (not an instance)
in case the user means to address it this way
Using the addrTo after it has failed its check means <invalid>/no
address will be printed. Change this throughout, but particularly
add a test for the origin issue for this.
* command/state list: list resources in nested and expaneded modules
A few distinct bugs fixed in here:
There was a bug in the logic checking if a given module was the child of
the targetAddr, now fixed. That resolved the basic issue where resources
in nested submodules were not listed.
The logic around allowMissing needed some tweaking to allow for empty
modules, as long as those modules had submodules with resources. state
list is the only command using allowMissing with false so this felt safe
to do.
Finally I extended the logic so list would included expanded modules,
which is to say giving module.foo would result in resources from
module.foo[1], module.foo[0], etc.
* update state list docs to show that module filtering includes any nested
modules
Expressions such as "path.root" were returning the cwd (or modulePath),
instead of the usual _relative_ path. This commit normalizes the path
before building the context.
Also uncomment and fix some tests which had been skipped for a couple of
years. Those validate cases work now!
Note that these test cases and the JSON output are not especially
minimized, making them snapshot/golden tests. The output looks correct
at time of writing, and we don't expect to change validate significantly
any time soon, but if we do there will be some churn here.
We included these warnings in v0.14 after noticing that we'd accidentally
published some incorrect documentation about the purpose of the plugin
cache directory under .terraform/plugins. We switched to using
.terraform/providers instead so that we could treat any missing providers
that appear in the legacy directory as likely to be a result of following
that documentation, and thus produce this extra warning.
However, the further we get from v0.13 the more likely it is for this
warning to be a confusing false positive rather than something helpful,
and this is a non-trivial codepath requiring us to retain a concept that
we otherwise don't need (the "legacy cache dir"), so here we'll remove
those warnings and support code for v0.15 onwards.
These warnings were always accompanied by an error message saying that a
provider could not be found, and that error message remains after this
change. This just removes the "by the way..."-style warning we had been
emitting alongside the errors.
If a user forgets to specify the source address for a provider, Terraform
will assume they meant a provider in the registry.terraform.io/hashicorp/
namespace. If that ultimately doesn't exist, we'll now try to see if
there's some other provider source address recorded in the registry's
legacy provider lookup table, and suggest it if so.
The error message here is a terse one addressed primarily to folks who are
already somewhat familiar with provider source addresses and how to
specify them. Terraform v0.13 had a more elaborate version of this error
message which directed the user to try the v0.13 automatic upgrade tool,
but we no longer have that available in v0.14 and later so the user must
make the fix themselves.
The upstream bug with opening a browser on Windows Subsystem for Linux
has been fixed, so this reverts our local patch for this. The approach
upstream adds fallback support for x-www-browser and www-browser if
xdg-open fails, and this fixes the problem on WSL.
This reverts commit 12e090ce48.
So far the output command has had a default output format intended for
human consumption and a JSON output format intended for machine
consumption.
However, until Terraform v0.14 the default output format for primitive
types happened to be _almost_ a raw string representation of the value,
and so users started using that as a more convenient way to access
primitive-typed output values from shell scripts, avoiding the need to
also use a tool like "jq" to decode the JSON.
Recognizing that primitive-typed output values are common and that
processing them with shell scripts is common, this commit introduces a new
-raw mode which is explicitly intended for that use-case, guaranteeing
that the result will always be the direct result of a string conversion
of the output value, or an error if no such conversion is possible.
Our policy elsewhere in Terraform is that we always use JSON for
machine-readable output. We adopted that policy because our other
machine-readable output has typically been complex data structures rather
than single primitive values. A special mode seems justified for output
values because it is common for root module output values to be just
strings, and so it's pragmatic to offer access to the raw value directly
rather than requiring a round-trip through JSON.
As of Terraform 0.13+, the get-plugins command has been
superceded by new provider installation mechanisms, and
general philosophy (providers are always installed, but
the sources may be customized). Updat the init command
to give users a warning if they are setting this flag,
to encourage them to remove it from their workflow, and
update relevant docs and docstrings as well
* command/format: concise diff is no longer an experiment
Since state formatting goes through the "diff" printer, I have
repurposed the concise flag as a verbose flag, used only when printing
state. It's silly but it works!
* remove helper/experiment
With this experiment concluded, we no longer need helper/experiment. The
shadow experiment had not been touched in many years, so I removed all
references, and removed the package entirely. Any new experiments are
expected to be configuration experiments handled by our (other)
experiments package.
* check for the verbose flag consistently, in case we end up using it in plans in the future
Along with all of the other information we previously reported in the
"terraform version" output, we'll now include the name of the current
platform as our provider mechanisms represent it.
This is addressing a long-standing minor annoyance where we often can't
tell from an incomplete bug report which platform Terraform was running
on, and incomplete bug reporters do tend to at least include the
"terraform version" output even if they don't also include the requested
full trace log.
However, what motivated doing it _now_ is that anyone building a provider
registry or mirror needs to have some awareness of these platform
identifiers which have been, until v0.13, mostly an implementation detail.
This additional information is a small thing we can do to help registry
builders find out what the platform identifier ought to be for each of
the platforms they aim to support, even if some of them are platforms
which the Go compiler allows but which HashiCorp doesn't officially
support.
The new information is on a line of its own in the output as a pragmatic
way to avoid breaking anyone who might be using something like
$(terraform version | head -n1) to print a brief Terraform version
identifier into some logs. That's not an interface we officially support
for machine consumption, but it's easy to avoid breaking it here and so we
won't do so.
When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.
This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.
To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.
Terraform version compatibility is defined as:
- For all releases before 0.14.0, local must exactly equal remote, as
two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
we will not change the state version number in a patch release.
If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.
Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.
In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
Remove chef, habitat, puppet, and salt-masterless provsioners,
which follows their deprecation. Update the documentatin for these
provisioners to clarify that they have been removed from later versions
of Terraform. Adds the fmt Make target back and updates fmtcheck script
for correctness.
When building a context, we read the dependency locks and ensure that
the provider requirements from the configuration can be satisfied.
If the configured requirements change such that the locks need to be
updated, we explain this and recommend running "terraform init".
This check is ignored for any providers which are locally marked as in
development. This includes unmanaged providers and those listed in the
provider installation `dev_overrides` block.
Core is only using the PrepareProviderConfig call for the validation
part of the method, but we should be re-validating the final config
immediately before Configure.
This change elects to not start using the PreparedConfig here, since
there is no useful reason for it at this point, and it would
introduce a functional difference between terraform releases that can be
avoided.
Previously when printing the relevant variables involved in a failed
expression evaluation we would just skip over unknown values entirely.
There are some errors, though, which are _caused by_ a value being
unknown, in which case it's helpful to show which of the inputs to that
expression were known vs. unknown so that the user can limit their further
investigation only to the unknown ones.
While here I also added a special case for sensitive values that overrides
all other display, because we don't know what about a value is sensitive
and so better to give nothing away at the expense of a slightly less
helpful error message.
For this version of Terraform and forward, we no longer refuse to read
compatible state files written by future versions of Terraform. This is
a commitment that any changes to the semantics or format of the state
file after this commit will require a new state file version 5.
The result of this is that users of this Terraform version will be able
to share remote state with users of future versions, and all users will
be able to read and write state. This will be true until the next major
state file version is required.
This does not affect users of previous versions of Terraform, which will
continue to refuse to read state written by later versions.
The short description of our commands (as shown in the main help output
from "terraform") was previously very inconsistent, using different
tense/mood for different commands. Some of the commands were also using
some terminology choices inconsistent with how we currently talk about
the related ideas in our documentation.
Here I've tried to add some consistency by first rewriting them all in
the imperative mood (except the ones that just are just subcommand
groupings), and tweaking some of the terminology to hopefully gel better
with how we present similar ideas in our recently-updated docs.
While working on this I inevitably spotted some similar inconsistencies
in the longer-form help output of some of the commands. I've not reviewed
all of these for consistency, but I did update some where the wording
was either left inconsstent with the short form changes I'd made or
where the prose stood out to me as particularly inconsistent with our
current usual documentation language style.
All of this is subjective, so I expect we'll continue to tweak these over
time as we continue to develop our documentation writing style based on
user questions and feedback.
Now that hclog can independently set levels on related loggers, we can
separate the log levels for different subsystems in terraform.
This adds the new environment variables, `TF_LOG_CORE` and
`TF_LOG_PROVIDER`, which each take the same set of log level arguments,
and only applies to logs from that subsystem. This means that setting
`TF_LOG_CORE=level` will not show logs from providers, and
`TF_LOG_PROVIDER=level` will not show logs from core. The behavior of
`TF_LOG` alone does not change.
While it is not necessarily needed since the default is to disable logs,
there is also a new level argument of `off`, which reflects the
associated level in hclog.
Use a separate log sink to always capture trace logs for the panicwrap
handler to write out in a crash log.
This requires creating a log file in the outer process and passing that
path to the child process to log to.
If you run the e2etests locally and use a configured plugin_cache_dir,
the test will leave a bad directory behind in your cache dir that causes
later `init`s to fail. To circumvent this, pass an explicity-empty CLI
config file.
This is a nicety for local developers and not necessarily required, but
it happens to me often enough that I'd like to fix it. It's probably not
a *bad* idea to pass an explicit cli config to all e2etests, honestly,
but this is the only one that causes active problems so I limited this
PR to that one test.
Here's the error which occurs on subsequent `init` if this test is run on a
machine that uses a plugin cache dir:
2020/10/13 10:41:05 [TRACE] providercache.fillMetaCache: error while scanning directory /Users/mildwonkey/.terraform.d/plugin-cache: failed to read metadata about /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: stat /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: no such file or directory
If a configuration requires a partial provider version (with some parts
unspecified), Terraform considers this as a constrained-to-zero version.
For example, a version constraint of 1.2 will result in an attempt to
install version 1.2.0, even if 1.2.1 is available.
When writing the dependency locks file, we previously would write 1.2.*,
as this is the in-memory representation of 1.2. This would then cause an
error on re-reading the locks file, as this is not a valid constraint
format.
Instead, we now explicitly convert the constraint to its zero-filled
representation before writing the locks file. This ensures that it
correctly round-trips.
Because this change is made in getproviders.VersionConstraintsString, it
also affects the output of the providers sub-command.
Use a single log writer instance for all std library logging.
Setup the std log writer in the logging package, and remove boilerplate
from test packages.
For normal provider installation we want to associate each provider with
a selected version number and find a suitable package for that version
that conforms to the official hashes for that release.
Those requirements are very onerous for a provider developer currently
testing a not-yet-released build, though. To allow for that case this new
CLI configuration feature allows overriding specific providers to refer
to give local filesystem directories.
Any provider overridden in this way is not subject to the usual
restrictions about selected versions or checksum conformance, and
activating an override won't cause any changes to the selections recorded
in the lock file because it's intended to be a temporary setting for one
developer only.
This is, in a sense, a spiritual successor of an old capability we had to
override specific plugins in the CLI configuration file. There were
some vestiges of that left in the main package and CLI config package
but nothing has actually been honoring them for several versions now and
so this commit removes them to avoid confusion with the new mechanism.
This originated in the cliconfig code to write out credentials files. The
Windows implementation of this in particular was quite onerous to get
right because it needs a very specific sequence of operations to avoid
running into exclusive file locks, and so by factoring this out with
only cosmetic modification we can avoid repeating all of that engineering
effort for other atomic file writing use-cases.
Terraform v0.10 introduced .terraform/plugins as a cache directory for
automatically-installed plugins, Terraform v0.13 later reorganized the
directory structure inside but retained its purpose as a cache.
The local cache used to also serve as a record of specifically which
packages were selected in a particular working directory, with the intent
that a second run of "terraform init" would always select the same
packages again. That meant that in some sense it behaved a bit like a
local filesystem mirror directory, even though that wasn't its intended
purpose.
Due to some unfortunate miscommunications, somewhere a long the line we
published some documentation that _recommended_ using the cache directory
as if it were a filesystem mirror directory when working with Terraform
Cloud. That was really only working as an accident of implementation
details, and Terraform v0.14 is now going to break that because the source
of record for the currently-selected provider versions is now the
public-facing dependency lock file rather than the contents of an existing
local cache directory on disk.
After some consideration of how to move forward here, this commit
implements a compromise that tries to avoid silently doing anything
surprising while still giving useful guidance to folks who were previously
using the unsupported strategy. Specifically:
- The local cache directory will now be .terraform/providers rather than
.terraform/plugins, because .terraform/plugins is effectively "poisoned"
by the incorrect usage that we can't reliably distinguish from prior
version correct usage.
- The .terraform/plugins directory is now the "legacy cache directory". It
is intentionally _not_ now a filesystem mirror directory, because that
would risk incorrectly interpreting providers automatically installed
by Terraform v0.13 as if they were a local mirror, and thus upgrades
and checksum fetches from the origin registry would be blocked.
- Because of the previous two points, someone who _was_ trying to use the
legacy cache directory as a filesystem mirror would see installation
fail for any providers they manually added to the legacy directory.
To avoid leaving that user stumped as to what went wrong, there's a
heuristic for the case where a non-official provider fails installation
and yet we can see it in the legacy cache directory. If that heuristic
matches then we'll produce a warning message hinting to move the
provider under the terraform.d/plugins directory, which is a _correct_
location for "bundled" provider plugins that belong only to a single
configuration (as opposed to being installed globally on a system).
This does unfortunately mean that anyone who was following the
incorrectly-documented pattern will now encounter an error (and the
aforementioned warning hint) after upgrading to Terraform v0.14. This
seems like the safest compromise because Terraform can't automatically
infer the intent of files it finds in .terraform/plugins in order to
decide automatically how best to handle them.
The internals of the .terraform directory are always considered
implementation detail for a particular Terraform version and so switching
to a new directory for the _actual_ cache directory fits within our usual
set of guarantees, though it's definitely non-ideal in isolation but okay
when taken in the broader context of this problem, where the alternative
would be silent misbehavior when upgrading.
When an attribute's sensitivity changes, but its value remains the same,
we consider this an update operation for the plan. This commit updates
the diff renderer to match this, detecting and displaying the change in
sensitivity.
Previously, the renderer would detect no changes to the value of the
attribute, and consider it a no-op action. This resulted in suppression
of the attribute when the plan is in concise mode.
This is achieved with a new helper function, ctyEqualValueAndMarks. We
call this function whenever we want to check that two values are equal
in order to determine whether the action is update or no-op.
Previously we were just letting hclwrite do its default formatting
behavior here. The current behavior there isn't ideal anyway -- it puts
big data structures all on one line -- but even ignoring that our goal
for this file format is to keep things in a highly-normalized shape so
that diffs against the file are clear and easy to read.
With that in mind, here we directly control how we write that value into
the file, which means that later changes to hclwrite's list/set
presentation won't affect it, regardless of what form they take.
This command is intended to help support situations where Terraform is
configured to use only local mirrors for provider installation and so the
normal "terraform init" flow would not have direct access to the official
package checksums published in the origin registry.
The intended workflow here is to use this command only when adding a new
provider or changing an existing provider's version in the configuration,
to augment the lock file with all of the checksums required to verify
the provider across a variety of different platforms. Once this command
has recorded all of the official checksums, future runs of
"terraform init" will verify that provider packages obtained from a local
mirror match with those upstream checksums.
This probably isn't the best UI we could do here, but it's a placeholder
for now just to avoid making it seem like we're ignoring the lock file
and checking for new versions anyway.
This changes the approach used by the provider installer to remember
between runs which selections it has previously made, using the lock file
format implemented in internal/depsfile.
This means that version constraints in the configuration are considered
only for providers we've not seen before or when -upgrade mode is active.
helper/copy CopyDir was used heavily in tests. It differes from
internal/copydir in a few ways, the main one being that it creates the
dst directory while the internal version expected the dst to exist
(there are other differences, which is why I did not just switch tests
to using internal's CopyDir).
I moved the CopyDir func from helper/copy into command_test.go; I could
also have moved it into internal/copy and named it something like
CreateDirAndCopy so if that seems like a better option please let me
know.
helper/copy/CopyFile was used in a couple of spots so I moved it into
internal, at which point I thought it made more sense to rename the
package copy (instead of copydir).
There's also a `go mod tidy` included.
Pam Selle