Commit Graph

204 Commits

Author SHA1 Message Date
James Bardin 8c5e11d41a add relevant_attributes to the json plan format
Add the resource instances and individual attributes which may have
contributed to the planned changes to the json format of the plan. We
use the existing path encoding for individual attributes, which is
already used in the replace_paths change field.
2022-03-17 09:35:36 -04:00
James Bardin f0cd8be66f add whole resource references 2022-03-17 09:35:36 -04:00
James Bardin e2b74247f2 track contributing instances
Track individual instance drift rather than whole resources which
contributed to the plan. This will allow the output to be more precise,
and we can still use NoKey instances as a proxy for containing resources
when needed.
2022-03-17 09:35:36 -04:00
James Bardin 25f4c0d3dd filter attribute refresh changes from plan UI
Filter the refresh changes from the normal plan UI at the attribute
level. We do this by constructing fake plans.Change records for diff
generation, reverting all attribute changes that do not match any of the
plan's ContributingResourceReferences.
2022-03-17 09:35:36 -04:00
James Bardin 0e7cec83db decode change before creating diff
This is functionally equivalent, but will allow us to filter the change
values directly for reduced drift output.
2022-03-17 09:35:36 -04:00
James Bardin c02e8bc5b3 change plan to store individual relevant attrs
Storing individual contributing attributes will allow finer tuning of
the plan rendering.

add contributing to outputs
2022-03-17 09:35:36 -04:00
James Bardin f1e8aed48d output drift footer in the right place 2022-03-04 15:51:36 -05:00
James Bardin c5c7045a89 filter out non-relevant drift changes
Only show drift changes which may have affected the plan output.
2022-03-04 15:51:36 -05:00
James Bardin a151aaef05 only show drift when there are changes 2022-03-04 15:51:36 -05:00
Alisdair McDiarmid 70db495d00
Merge pull request #30525 from nozaq/provider-config-default
jsonconfig: fix keys for default providers
2022-02-22 16:13:41 -05:00
Alisdair McDiarmid 3e4d6b252f jsonplan: Improve performance for deep objects
When calculating the unknown values for JSON plan output, we would
previously recursively call the `unknownAsBool` function on the current
sub-tree twice, if any values were unknown. This was wasteful, but not
noticeable for normal Terraform resource shapes.

However for deeper nested object values, such as Kubernetes manifests,
this was a severe performance problem, causing `terraform show -json` to
take several hours to render a plan.

This commit reuses the already calculated unknown value for the subtree,
and adds benchmark coverage to demonstrate the improvement.
2022-02-18 17:00:18 -05:00
nozaq 3c32f7a56c
jsonconfig: add implicitly created provider configs 2022-02-19 01:55:09 +09:00
nozaq 0ce040405c
jsonconfig: fix provider mappings with same names 2022-02-19 00:44:48 +09:00
nozaq 7f4019e1f6
jsonconfig: fix keys for default providers
Fixes provider config keys to reflect implicit provider inheritance.
2022-02-16 01:38:57 +09:00
Pedro Belém aed7162e9a
cli: Fix state mv exit code for missing resource (#29839) 2022-02-14 15:20:03 -05:00
Alisdair McDiarmid fe8183c4af json: Increment JSON plan format version
The JSON plan configuration data now includes a `full_name` field for
providers. This addition warrants a backwards compatible increment to
the version number.
2022-02-07 15:06:05 -05:00
Alisdair McDiarmid ddc81a204f json: Disregard format version in tests
Instead of manually updating every JSON output test fixture when we
change the format version, disregard any differences when testing.
2022-02-07 15:05:58 -05:00
Alisdair McDiarmid f5b90f84a8 jsonconfig: Improve provider configuration output
When rendering configuration as JSON, we have a single map of provider
configurations at the top level, since these are globally applicable.
Each resource has an opaque key into this map which points at the
configuration data for the provider.

This commit fixes two bugs in this implementation:

- Resources in non-root modules had an invalid provider config key,
  which meant that there was never a valid reference to the provider
  config block. These keys were prefixed with the local module name
  instead of the path to the module. This is now corrected.

- Modules with passed provider configs would point to either an empty
  provider config block or one which is not present at all. This has
  been fixed so that these resources point to the provider config block
  from the calling module (or wherever up the module tree it was
  originally defined).

We also add a "full_name" key-value pair to the provider config block,
with the entire fully-qualified provider name including hostname and
namespace.
2022-02-07 15:05:58 -05:00
Alisdair McDiarmid 5a9bc76d1a
Merge pull request #30316 from superkooks/main
Fix autocomplete for workspace subcommands
2022-01-21 12:01:41 -05:00
Krista LaFentres (she/her) 6dcf00aefc
Merge pull request #30344 from hashicorp/lafentres/refactor-show-command
cli: Refactor show command & migrate to command arguments and views
2022-01-13 13:58:53 -06:00
Krista LaFentres fea8f6cfa2 cli: Migrate show command to use command arguments and views 2022-01-13 11:00:03 -06:00
Krista LaFentres 8d1bced812 cli: Refactor show command to remove dependence on local run and only load the backend when we need it
See https://github.com/hashicorp/terraform/pull/30205#issuecomment-997113175 for more context
2022-01-12 13:47:59 -06:00
Martin Atkins 36c4d4c241 core and backend: remove redundant handling of default variable values
Previously we had three different layers all thinking they were
responsible for substituting a default value for an unset root module
variable:
 - the local backend, via logic in backend.ParseVariableValues
 - the context.Plan function (and other similar functions) trying to
   preprocess the input variables using
   terraform.mergeDefaultInputVariableValues .
 - the newer prepareFinalInputVariableValue, which aims to centralize all
   of the variable preparation logic so it can be common to both root and
   child module variables.

The second of these was also trying to handle type constraint checking,
which is also the responsibility of the central function and not something
we need to handle so early.

Only the last of these consistently handles both root and child module
variables, and so is the one we ought to keep. The others are now
redundant and are causing prepareFinalInputVariableValue to get a slightly
corrupted view of the caller's chosen variable values.

To rectify that, here we remove the two redundant layers altogether and
have unset root variables pass through as cty.NilVal all the way to the
central prepareFinalInputVariableValue function, which will then handle
them in a suitable way which properly respects the "nullable" setting.

This commit includes some test changes in the terraform package to make
those tests no longer rely on the mergeDefaultInputVariableValues logic
we've removed, and to instead explicitly set cty.NilVal for all unset
variables to comply with our intended contract for PlanOpts.SetVariables,
and similar. (This is so that we can more easily catch bugs in callers
where they _don't_ correctly handle input variables; it allows us to
distinguish between the caller explicitly marking a variable as unset vs.
not describing it at all, where the latter is a bug in the caller.)
2022-01-10 12:26:54 -08:00
SuperKooks 5171c89e1e Fix autocomplete for workspace subcommands 2022-01-08 14:41:20 +11:00
Martin Atkins b802db75d7 build: Build and run e2etest as part of the release build pipeline
This uses the decoupled build and run strategy to run the e2etests so that
we can arrange to run the tests against the real release packages produced
elsewhere in this workflow, rather than ones generated just in time by
the test harness.

The modifications to make-archive.sh here make it more consistent with its
originally-intended purpose of producing a harness for testing "real"
release executables. Our earlier compromise of making it include its own
terraform executable came from a desire to use that script as part of
manual cross-platform testing when we weren't yet set up to support
automation of those tests as we're doing here. That does mean, however,
that the terraform-e2etest package content must be combined with content
from a terraform release package in order to produce a valid contest for
running the tests.

We use a single job to cross-compile the test harness for all of the
supported platforms, because that build is relatively fast and so not
worth the overhead of matrix build, but then use a matrix build to
actually run the tests so that we can run them in a worker matching the
target platform.

We currently have access only to amd64 (x64) runners in GitHub Actions
and so for the moment this process is limited only to the subset of our
supported platforms which use that architecture.
2022-01-05 14:31:04 -08:00
Alisdair McDiarmid 535da4ebc7
Merge pull request #30205 from hashicorp/alisdair/fix-show-plan-against-non-default-state
command/show: Disable plan state lineage checks
2022-01-05 12:03:28 -05:00
Martin Atkins 23395a1022 providercache: Discard lock entries for unused providers
Previously we would only ever add new lock entries or update existing
ones. However, it's possible that over time a module may _cease_ using
a particular provider, at which point we ought to remove it from the lock
file so that operations won't fail when seeing that the provider cache
directory is inconsistent with the lock file.

Now the provider installer (EnsureProviderVersions) will remove any lock
file entries that relate to providers not included in the given
requirements, which therefore makes the resulting lock file properly match
the set of packages the installer wrote into the cache.

This does potentially mean that someone could inadvertently defeat the
lock by removing a provider dependency, running "terraform init", then
undoing that removal, and finally running "terraform init" again. However,
that seems relatively unlikely compared to the likelihood of removing
a provider and keeping it removed, and in the event it _did_ happen the
changes to the lock entry for that provider would be visible in the diff
of the provider lock file as usual, and so could be noticed in code
review just as for any other change to dependencies.
2021-12-17 15:30:21 -08:00
Alisdair McDiarmid 768741c0f7 command/show: Disable plan state lineage checks
When showing a saved plan, we do not need to check the state lineage
against current state, because the plan cannot be applied. This is
relevant when plan and apply specify a `-state` argument to choose a
non-default state file. In this case, the stored prior state in the plan
will not match the default state file, so a lineage check will always
error.
2021-12-17 17:46:42 -05:00
Chris Arcand 8b8fe2771f
Merge pull request #30142 from hashicorp/chrisarcand/remote-backend-no-workspaces-regression
command/meta_backend: Allow the remote backend to have no workspaces [again]
2021-12-14 09:58:50 -06:00
Chris Arcand 98978b3853 command/meta_backend: Allow the remote backend to have no workspaces [again]
A regression introduced in d72a413ef8

The comment explains, but TLDR: The remote backend actually *depended*
on being able to write it's backend state even though an 'error'
occurred (no workspaces).
2021-12-14 09:50:42 -06:00
Martin Atkins 096cddb4b7 command/format: Limitation of plans.ResourceInstanceDeleteBecauseNoModule
This is an explicit technical debt note that our plan renderer isn't able
to give a fully-specific hint in this particular case of deletion reason.

This reason code means that at least one of the module instance keys in
the resource's module path doesn't match an instance declared in the
configuration, but the plan data structure doesn't retain enough
information to know which is the first step in the path which refers to
a missing instance, and so we just always return the whole thing.

This would be confusing if we return module.foo[0].module.bar not being
in the configuration as a result of module.foo not using "count"; it would
be better to say "module.foo[0] is not in the configuration" instead.

It would be most ideal to handle all of the different situations that
ResourceInstanceDeleteBecauseWrongRepetition's rendering does, so that we
can go further and explain exactly _why_ that module instance isn't
declared anymore.

We can do neither of those things today because only the Terraform Core
"expander" component knows that information, and we've discarded that
by the time we get to rendering a plan. To fix this one day would require
preserving in the plan information about which module instances are
declared, as a separate sidecar data structure from which resource
instances we're taking actions on, and then using that to identify which
step in addr.Module here first selects an invalid instance.
2021-12-13 10:04:15 -05:00
Chris Arcand f4f5b7c968
Merge pull request #30059 from hashicorp/barrettclark/explicit-local-empty-migrate-messaging
Cloud: Do not ask to migrate empty default workspace
2021-12-01 11:53:36 -06:00
Barrett Clark c706c8f92b Do not ask to migrate empty default workspace
When migrating from an explicit local backend to Terraform Cloud, we ask
if you want to migrate the state. If there is no state to migrate we
should not ask if they want to migrate the emptiness.
2021-12-01 11:43:41 -06:00
Barrett Clark e08a02e7bf Fixes Issue #29959, Apply w/o init error message
When going from a local backend to Terraform Cloud, if you skip the
`terraform init` and run `terraform apply` this will give the user more
clear instructions.
2021-12-01 11:28:35 -06:00
Luces Huayhuaca d72a413ef8
command/meta_backend: Prompt to select workspace before saving backend config (#29945)
When terraform detects that a user has no workspaces that map to their current configuration, it will prompt the user to create a new workspace and enter a value name. If the user ignores the prompt and exits it, the legacy backend (terraform.tfstate) will be left in a awkward state:

1. This saved backend config will show a diff for the JSON attributes "serial", "tags" and "hash"
2. "Terraform workspace list" will show an empty list
3. "Terraform apply" will run successfully using the previous workspace, from the previous config, not the one from the current saved backend config
4. The cloud config is not reflective of the current working directory

Solution: If the user exits the prompt, the saved backend config should not be updated because they did not select a new workspace. They are back at the beginning where they are force to re run the init cmd again before proceeding with new changes.
2021-12-01 08:53:47 -08:00
Chris Arcand b5af7b6c92
Merge pull request #29987 from hashicorp/chrisarcand/backend-flag-with-tfc
command/init: Add -cloud alias to -backend, adjust `init` help output
2021-11-29 08:06:08 -06:00
Barrett Clark a6b56ad76f
Merge pull request #29997 from hashicorp/barrettclark/cloud-input-false
Cloud integration requires input for migrations
2021-11-23 16:22:27 -06:00
Barrett Clark 419676cb69 Cloud integration requires input for migrations
We cannot programmatically migrate workspaces to Terraform Cloud without
prompts, so `-input=false` should not be allowed in those cases.

There are 4 scenarios where we need input from a user to complete
migrating workspaces to Terraform Cloud.

1.) Migrate from a single local workspace to Terraform Cloud

* Terraform config for a local backend. Implicit local (no backend
  specified) is fine.
* `terraform init` and `terraform apply`
* Change the Terraform config to use the cloud block
* `terraform init -input=false`
* You should now see an error message

2.) Migrate from a remote backend with a prefix to Terraform Cloud with
  tags

* Create a workspace in Terraform Cloud manually. The name should
  include a prefix, like "app-one"
* Have the terraform config use `backend "remote"` with a prefix set to
  "app-"
* `terraform init` and `terraform apply`
* Update the Terraform config to use a cloud block with `tags
  = ["app"]`. There should not be a prefix defined in the config now.
* `terraform init -input=false`
* You should now see an error message

3.) Migrate from multiple local workspaces to a single Terraform Cloud
  workspace
* Create one or many local workspaces
* `terraform init` and `terraform apply` in each
* Change the Terraform config to use the cloud block
* `terraform init -input=false`
* You should now see an error message

4.) Migrate to Terraform Cloud and ask for a workspace name
* Create several local workspaces
* `terraform init` and `terraform apply` in each
* Change the Terraform config to use the cloud block with tags
* `terraform init -input=false`
* You should now see an error message
2021-11-23 10:39:42 -06:00
Luces Huayhuaca f63b6198ca
Create a function for logic that assigns value to initReason var after changing backend configuration (#29967)
* Create a function for logic that assigns value to initReason var after changing backend configuration

Create func determineInitReason() for logic block that assigns value to initReason var after changing backend/cloud configuration block or migrating to a different type of backend configuration. Also clarify 'cloud' configuration block message to say 'Terraform Cloud configuration block has changed' instead of 'Terraform Cloud configuration has changed'.
2021-11-22 13:32:34 -08:00
Alisdair McDiarmid a8b9d086b2 cli: Fix nested single and map diff rendering
Indentation was out by two spaces. This is now consistent with the other
displays of object/map values: four spaces for each nesting level.
2021-11-19 15:48:10 -05:00
Chris Arcand a0c7c1dc8d command/init: Adjust init help output
Some of the wording here needed adjusting with the change that backends
largely reflect state snapshot storage (removing 'enhanced'
designation), and that a 'backend' is not necessarily always present.
2021-11-19 14:40:12 -06:00
Chris Arcand 33d7c1e6d6 command/init: Add -cloud alias (of -backend)
This fixes an issue where a user could not disable initialization of the
'cloud' configuration block (As is possible with -backend=false), as
well as add some syntactic sugar around -backend by adding a mutually
exclusive -cloud alias.
2021-11-19 14:40:12 -06:00
Alisdair McDiarmid b562c4fb84 cli: Fix nested set diff rendering indentation 2021-11-19 13:06:34 -05:00
Alisdair McDiarmid 2d0349d9a4 cli: Fix diff for nested set unchanged elements
Unchanged elements in nested attributes backed by sets were previously
misrendered as empty objects. This commit removes the additional
brackets and adds a count of unchanged elements.
2021-11-19 11:53:36 -05:00
Martin Atkins 7d6d31eff8 command/init: Skip redundant state migration prompt in Cloud mode
The specialized Terraform Cloud migration process asks right up top
whether the user wants to migrate state, because there are various other
questions contingent on that answer.

Therefore we ought to just honor their earlier answer when we get to the
point of actually doing the state migration, rather than prompting again.

This is tricky because we're otherwise just reusing a codepath that's
common to both modes. Hopefully we can find a better way to do this in
a later commit, but for the moment our main motivation is minimizing risk
to the very next release.
2021-11-17 14:20:44 -08:00
Martin Atkins bac59d2480 command/init: Be explicit that some options are not relevant for Cloud
There are a few command line options for "terraform init" which are only
relevant when working with traditional backends, with the Cloud
integration previously just mostly ignoring them, or sometimes misbehaving
slightly due to them creating an unreasonable situation.

Now we'll catch these and return explicit errors, in order to be clear
that these options are not needed nor supported in Cloud mode.
2021-11-17 14:20:44 -08:00
Barrett Clark a146a2746e Add clarifying commend and e2e test
This pull request focuses on removing the prompt to rename the default
workspace when it is empty. Functionality already exists to not migrate
an empty workspace. This commit adds some clarifying language in the
comment where we do the evaluation to know whether to ask for a new name
or not. I also added an end to end test, which I should have added to
begin with.
2021-11-16 13:05:26 -06:00
Barrett Clark a9eb62d692 Cloud: Migrate empty default workspace prompt
Given: You have multiple explicit local workspaces, and the `default`
workspace is empty.
When: You migrate the workspaces to Terraform Cloud.
Then: Terraform should _not_ ask for a workspace to migrate the
`default` workspace to in Terraform Cloud.
2021-11-16 10:33:46 -06:00
Chris Arcand bf76cc98ef command: Suggestions on migration copy from user feedback
Some small edits to the TFC migration copy.
2021-11-15 09:21:31 -06:00
kmoe 64635edfb9
Merge pull request #29885 from hashicorp/kmoe/more-init-interrupts
command: make module installation interruptible
2021-11-11 12:37:49 +00:00