Commit Graph

814 Commits

Author SHA1 Message Date
Sebastian Rivera afb956d745
Merge pull request #30141 from hashicorp/preapply-runtasks-clioutput
Cloud run tasks (post-plan only) CLI integration
2022-02-25 15:46:46 -05:00
Sebastian Rivera 52c5f9f6b7 Updated for latest go-tfe run task changes 2022-02-25 15:32:16 -05:00
Sebastian Rivera 126d6df088 Added run task support for post plan run stage, removed pre apply
This commit stems from the change to make post plan the default run task stage, at the
time of this commit's writing! Since pre apply is under internal revision, we have removed
the block that polls the pre apply stage until the team decides to re-add support for pre apply
run tasks.
2022-02-24 14:06:57 -05:00
uturunku1 383da4893b use new enum string for task stages 2022-02-24 14:06:57 -05:00
Brandon Croft aa0dda81b4 Fall back to reading latest run without task_stages
Older versions of TFE will not allow "task_stages" as an include parameter. In this case, fall back to reading the Run without additional options.
2022-02-24 14:06:57 -05:00
uturunku1 a9da859ee5 rename variables to something more descriptive 2022-02-24 14:03:02 -05:00
uturunku1 77946af472 pull latest changes from go-tfe branch and use use new field name that previously was incorrectly named TaskStage 2022-02-24 14:03:00 -05:00
uturunku1 8090b23db7 delete unused function 2022-02-24 14:02:37 -05:00
Brandon Croft 0b8bb29a61 [cloud] refactor integration context and add code documentation 2022-02-24 14:02:37 -05:00
Brandon Croft 791c36c504 [cloud] report run tasks by name instead of assuming pre_apply 2022-02-24 14:02:37 -05:00
uturunku1 6b5da4d43c [cloud] run tasks output formatting 2022-02-24 14:02:37 -05:00
Brandon Croft 2938ec43fa [cloud] handle unreachable run tasks 2022-02-24 14:02:37 -05:00
uturunku1 3e9ae69a12 [cloud] run tasks integration
This change will await the completion of pre-apply run tasks if they
exist on a run and then report the results.

It also adds an abstraction when interacting with cloud integrations such
as policy checking and cost estimation that simplify and unify output,
although I did not go so far as to refactor those callers to use it yet.
2022-02-24 14:02:35 -05:00
Alisdair McDiarmid 70db495d00
Merge pull request #30525 from nozaq/provider-config-default
jsonconfig: fix keys for default providers
2022-02-22 16:13:41 -05:00
Alisdair McDiarmid 3e4d6b252f jsonplan: Improve performance for deep objects
When calculating the unknown values for JSON plan output, we would
previously recursively call the `unknownAsBool` function on the current
sub-tree twice, if any values were unknown. This was wasteful, but not
noticeable for normal Terraform resource shapes.

However for deeper nested object values, such as Kubernetes manifests,
this was a severe performance problem, causing `terraform show -json` to
take several hours to render a plan.

This commit reuses the already calculated unknown value for the subtree,
and adds benchmark coverage to demonstrate the improvement.
2022-02-18 17:00:18 -05:00
nozaq 3c32f7a56c
jsonconfig: add implicitly created provider configs 2022-02-19 01:55:09 +09:00
nozaq 0ce040405c
jsonconfig: fix provider mappings with same names 2022-02-19 00:44:48 +09:00
kmoe 161374725c
Warn when ignore_changes includes a Computed attribute (#30517)
* ignore_changes attributes must exist in schema

Add a test verifying that attempting to add a nonexistent attribute to
ignore_changes throws an error.

* ignore_changes cannot be used with Computed attrs

Return a warning if a Computed attribute is present in ignore_changes,
unless the attribute is also Optional.

ignore_changes on a non-Optional Computed attribute is a no-op, so the user
likely did not want to set this in config.
An Optional Computed attribute, however, is still subject to ignore_changes
behaviour, since it is possible to make changes in the configuration that
Terraform must ignore.
2022-02-18 10:38:29 +00:00
nozaq 7f4019e1f6
jsonconfig: fix keys for default providers
Fixes provider config keys to reflect implicit provider inheritance.
2022-02-16 01:38:57 +09:00
Pedro Belém aed7162e9a
cli: Fix state mv exit code for missing resource (#29839) 2022-02-14 15:20:03 -05:00
Alisdair McDiarmid d8018270a7
Merge pull request #30138 from hashicorp/alisdair/json-module-call-providers-mapping
jsonconfig: Improve provider configuration output
2022-02-10 10:33:47 -05:00
Alisdair McDiarmid 843c50e8ce lang: Further limit the console-only type function
This commit introduces a capsule type, `TypeType`, which is used to
extricate type information from the console-only `type` function. In
combination with the `TypeType` mark, this allows us to restrict the use
of this function to top-level display of a value's type. Any other use
of `type()` will result in an error diagnostic.
2022-02-10 06:12:58 -05:00
Alisdair McDiarmid 903d6f1055 lang: Remove use of marks.Raw in tests
These instances of marks.Raw usage were semantically only testing the
properties of combining multiple marks. Testing this with an arbitrary
value for the mark is just as valid and clearer.
2022-02-09 17:43:54 -05:00
Alisdair McDiarmid 691b98b612 cli: Prevent overuse of console-only type function
The console-only `type` function allows interrogation of any value's
type.  An implementation quirk is that we use a cty.Mark to allow the
console to display this type information without the usual HCL quoting.
For example:

> type("boop")
string

instead of:

> type("boop")
"string"

Because these marks can propagate when used in complex expressions,
using the type function as part of a complex expression could result in
this "print as raw" mark being attached to a collection. When this
happened, it would result in a crash when we tried to iterate over a
marked value.

The `type` function was never intended to be used in this way, which is
why its use is limited to the console command. Its purpose was as a
pseudo-builtin, used only at the top level to display the type of a
given value.

This commit goes some way to preventing the use of the `type` function
in complex expressions, by refusing to display any non-string value
which was marked by `type`, or contains a sub-value which was so marked.
2022-02-09 17:43:54 -05:00
Alisdair McDiarmid fe8183c4af json: Increment JSON plan format version
The JSON plan configuration data now includes a `full_name` field for
providers. This addition warrants a backwards compatible increment to
the version number.
2022-02-07 15:06:05 -05:00
Alisdair McDiarmid ddc81a204f json: Disregard format version in tests
Instead of manually updating every JSON output test fixture when we
change the format version, disregard any differences when testing.
2022-02-07 15:05:58 -05:00
Alisdair McDiarmid f5b90f84a8 jsonconfig: Improve provider configuration output
When rendering configuration as JSON, we have a single map of provider
configurations at the top level, since these are globally applicable.
Each resource has an opaque key into this map which points at the
configuration data for the provider.

This commit fixes two bugs in this implementation:

- Resources in non-root modules had an invalid provider config key,
  which meant that there was never a valid reference to the provider
  config block. These keys were prefixed with the local module name
  instead of the path to the module. This is now corrected.

- Modules with passed provider configs would point to either an empty
  provider config block or one which is not present at all. This has
  been fixed so that these resources point to the provider config block
  from the calling module (or wherever up the module tree it was
  originally defined).

We also add a "full_name" key-value pair to the provider config block,
with the entire fully-qualified provider name including hostname and
namespace.
2022-02-07 15:05:58 -05:00
Alisdair McDiarmid 639eb5212f core: Remove unused PlanOpts.Validate
This vestigial field was written to but never read.
2022-02-03 14:16:25 -05:00
Alisdair McDiarmid 7ded73f266 configs: Validate pre/postcondition self-refs
Preconditions and postconditions for resources and data sources may not
refer to the address of the containing resource or data source. This
commit adds a parse-time validation for this rule.
2022-02-03 09:37:22 -05:00
Alisdair McDiarmid cdae6d4396 core: Add context tests for pre/post conditions 2022-01-31 15:38:26 -05:00
Alisdair McDiarmid a95ad997e1 core: Document postconditions as valid use of self
This is not currently gated by the experiment only because it is awkward
to do so in the context of evaluationStateData, which doesn't have any
concept of experiments at the moment.
2022-01-31 14:34:35 -05:00
Martin Atkins 5573868cd0 core: Check pre- and postconditions for resources and output values
If the configuration contains preconditions and/or postconditions for any
objects, we'll check them during evaluation of those objects and generate
errors if any do not pass.

The handling of post-conditions is particularly interesting here because
we intentionally evaluate them _after_ we've committed our record of the
resulting side-effects to the state/plan, with the intent that future
plans against the same object will keep failing until the problem is
addressed either by changing the object so it would pass the precondition
or changing the precondition to accept the current object. That then
avoids the need for us to proactively taint managed resources whose
postconditions fail, as we would for provisioner failures: instead, we can
leave the resolution approach up to the user to decide.

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2022-01-31 14:02:53 -05:00
Martin Atkins c827c049fe terraform: Precondition and postcondition blocks generate dependencies
If a resource or output value has a precondition or postcondition rule
then anything the condition depends on is a dependency of the object,
because the condition rules will be evaluated as part of visiting the
relevant graph node.
2022-01-28 11:00:29 -05:00
Martin Atkins 9076400436 configs: Decode preconditions and postconditions
This allows precondition and postcondition checks to be declared for
resources and output values as long as the preconditions_postconditions
experiment is enabled.

Terraform Core doesn't currently know anything about these features, so
as of this commit declaring them does nothing at all.
2022-01-28 11:00:29 -05:00
Martin Atkins 82c518209d experiments: New "preconditions_postconditions" experiment 2022-01-28 11:00:29 -05:00
Martin Atkins 4f41a0a1fe configs: Generalize "VariableValidation" as "CheckRule"
This construct of a block containing a condition and an error message will
be useful for other sorts of blocks defining expectations or contracts, so
we'll give it a more generic name in anticipation of it being used in
other situations.
2022-01-28 11:00:29 -05:00
Alisdair McDiarmid e21d5d36f4 core: Fix plan write/state write ordering bug 2022-01-26 15:33:18 -05:00
Alisdair McDiarmid 5a9bc76d1a
Merge pull request #30316 from superkooks/main
Fix autocomplete for workspace subcommands
2022-01-21 12:01:41 -05:00
Brian Flad a477d10bd1
Introduce Terraform Plugin Protocol 6.2 with legacy_type_system fields from Protocol 5 (#30375)
Reference: https://github.com/hashicorp/terraform/issues/30373

This change forward ports the `legacy_type_system` boolean fields in the `ApplyResourceChange.Response` and `PlanResourceChange.Response` messages that existed in protocol version 5, so that existing terraform-plugin-sdk/v2 providers can be muxed with protocol version 6 providers (e.g. terraform-plugin-framework) while also taking advantage of the newer protocol features. This functionality should not be used by any providers or SDKs except those built with terraform-plugin-sdk.

Updated via:

```shell
cp docs/plugin-protocol/tfplugin6.1.proto docs/plugin-protocol/tfplugin6.2.proto
# Copy legacy_type_system fields from tfplugin5.2.proto into ApplyResourceChange.Response and PlanResourceChange
rm internal/tfplugin6/tfplugin6.proto
ln -s ../../docs/plugin-protocol/tfplugin6.2.proto internal/tfplugin6/tfplugin6.proto
go run tools/protobuf-compile/protobuf-compile.go `pwd`
# Updates to internal/plugin6/grpc_provider.go
```
2022-01-20 09:57:42 -05:00
Krista LaFentres (she/her) 6dcf00aefc
Merge pull request #30344 from hashicorp/lafentres/refactor-show-command
cli: Refactor show command & migrate to command arguments and views
2022-01-13 13:58:53 -06:00
Krista LaFentres 64e1241ae3 backend/local: Remove unused DisablePlanFileStateLineageChecks flag
Now that show command has been refactored to remove its dependence
on a local backend and local run, this flag is no longer needed to
fix #30195.
2022-01-13 11:00:10 -06:00
Krista LaFentres fea8f6cfa2 cli: Migrate show command to use command arguments and views 2022-01-13 11:00:03 -06:00
Krista LaFentres 8d1bced812 cli: Refactor show command to remove dependence on local run and only load the backend when we need it
See https://github.com/hashicorp/terraform/pull/30205#issuecomment-997113175 for more context
2022-01-12 13:47:59 -06:00
Martin Atkins e95f29bf9d lang/funcs: fileexists slightly better "not a file" error message
Previously we were just returning a string representation of the file mode,
which spends more characters on the irrelevant permission bits that it
does on the directory entry type, and is presented in a Unix-centric
format that likely won't be familiar to the user of a Windows system.

Instead, we'll recognize a few specific directory entry types that seem
worth mentioning by name, and then use a generic message for the rest.

The original motivation here was actually to deal with the fact that our
tests for this function were previously not portable due to the error
message leaking system-specific permission detail that are not relevant
to the test. Rather than just directly addressing that portability
problem, I took the opportunity to improve the error messages at the same
time.

However, because of that initial focus there are only actually tests here
for the directory case. A test that tries to test any of these other file
modes would not be portable and in some cases would require superuser
access, so we'll just leave those cases untested for the moment since they
weren't tested before anyway, and so we've not _lost_ any test coverage
here.
2022-01-11 08:46:29 -08:00
Martin Atkins bdc5f152d7 refactoring: Implied move statements can be cross-package
Terraform uses "implied" move statements to represent the situation where
it automatically handles a switch from count to no-count on a resource.
Because that situation requires targeting only a specific resource
instance inside a specific module instance, implied move statements are
always presented as if they had been declared in the root module and then
traversed through the exact module instance path to reach the target
resource.

However, that means they can potentially cross a module package boundary,
if the changed resource belongs to an external module. Normally we
prohibit that to avoid the root module depending on implementation details
of the called module, but Terraform generates these implied statements
based only on information in the called module and so there's no need to
apply that same restriction to implied move statements, which will always
have source and destination addresses belonging to the same module
instance.

This change therefore fixes a misbehavior where Terraform would reject
an attempt to switch from no-count to count in a called module, where
previously the author of the calling configuration had no recourse to fix
it because the change has actually happened upstream.
2022-01-11 08:43:57 -08:00
James Bardin 684ed7505d remove synthetic default expression for variables
Now that variable evaluation checks for a nil expression the graph
transformer does not need to generate a synthetic expression for
variable defaults. This means that all default handling is now located
in one place, and we are not surprised by a configuration expression
showing up which doesn't actually exist in the configuration.

Rename nodeModuleVariable.evalModuleCallArgument to evalModuleVariable.
This method is no longer handling only the module call argument, it is
also dealing with the variable declaration defaults and validation
statements.

Add an additional tests for validation with a non-nullable variable.
2022-01-10 16:22:33 -05:00
Martin Atkins 9ebc3e1cd2 core: More accurate error message for invalid variable values
In earlier Terraform versions we had an extra validation step prior to
the graph walk which tried to partially validate root module input
variable values (just checking their type constraints) and then return
error messages which specified as accurately as possible where the value
had originally come from.

We're now handling that sort of validation exclusively during the graph
walk so that we can share the main logic between both root module and
child module variable values, but previously that shared code wasn't
able to generate such specific information about where the values had
originated, because it was adapted from code originally written to only
deal with child module variables.

Here then we restore a similar level of detail as before, when we're
processing root module variables. For child module variables, we use
synthetic InputValue objects which state that the value was declared
in the configuration, thus causing us to produce a similar sort of error
message as we would've before which includes a source range covering
the argument expression in the calling module block.
2022-01-10 12:26:54 -08:00
Martin Atkins 36c4d4c241 core and backend: remove redundant handling of default variable values
Previously we had three different layers all thinking they were
responsible for substituting a default value for an unset root module
variable:
 - the local backend, via logic in backend.ParseVariableValues
 - the context.Plan function (and other similar functions) trying to
   preprocess the input variables using
   terraform.mergeDefaultInputVariableValues .
 - the newer prepareFinalInputVariableValue, which aims to centralize all
   of the variable preparation logic so it can be common to both root and
   child module variables.

The second of these was also trying to handle type constraint checking,
which is also the responsibility of the central function and not something
we need to handle so early.

Only the last of these consistently handles both root and child module
variables, and so is the one we ought to keep. The others are now
redundant and are causing prepareFinalInputVariableValue to get a slightly
corrupted view of the caller's chosen variable values.

To rectify that, here we remove the two redundant layers altogether and
have unset root variables pass through as cty.NilVal all the way to the
central prepareFinalInputVariableValue function, which will then handle
them in a suitable way which properly respects the "nullable" setting.

This commit includes some test changes in the terraform package to make
those tests no longer rely on the mergeDefaultInputVariableValues logic
we've removed, and to instead explicitly set cty.NilVal for all unset
variables to comply with our intended contract for PlanOpts.SetVariables,
and similar. (This is so that we can more easily catch bugs in callers
where they _don't_ correctly handle input variables; it allows us to
distinguish between the caller explicitly marking a variable as unset vs.
not describing it at all, where the latter is a bug in the caller.)
2022-01-10 12:26:54 -08:00
Martin Atkins 37b1413ab3 core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.

This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.

To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
 - The "raw value" (exactly as given by the user) lives only in the graph
   node representing the variable, which mirrors how the _expression_
   for a child module variable lives in its graph node. This means that
   the flow for the two is the same except that there's no expression
   evaluation step for root module variables, because they arrive as
   constant values from the caller.
 - The set of variable values in the EvalContext is always only "final"
   values, after type conversion is complete. That in turn means we no
   longer need to do "just in time" conversion in
   evaluationStateData.GetInputVariable, and can just return the value
   exactly as stored, which is consistent with how we handle all other
   references between objects.

This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.

While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2022-01-10 12:26:54 -08:00
Martin Atkins 483c38aca1 core: Remove TestContext2Validate_PlanGraphBuilder
This test seems to be a holdover from the many-moons-ago switch from one
graph for all operations to separate graphs for plan and apply. It is
effectively just a copy of a subset of the content of the Context.Validate
function and is a maintainability hazard because it tends to lag behind
updates to that function unless changes there happen to make it fail.

This test doesn't cover anything that the other validate context tests
don't exercise as an implementation detail of calling Context.Validate,
so I've just removed it with no replacement.
2022-01-10 12:26:54 -08:00