Commit Graph

28395 Commits

Author SHA1 Message Date
Martin Atkins 89b2405080 lang/funcs: "sensitive" and "nonsensitive" functions
These aim to allow hinting to Terraform about situations where it's not
able to automatically infer value sensitivity.

"nonsensitive" is for situations where Terraform's behavior is too
conservative, such as when a new value is derived from a sensitive value
in such a way that all of the sensitive content is removed.

"sensitive", on the other hand, is for situations where Terraform can't
otherwise infer that a value is sensitive. These situations should be
pretty rare in a module that's making effective use of sensitive input
variables and output values, but the documentation shows one example of
an uncommon situation where a more direct hint via this function would
be needed.

Both of these functions are aimed at only occasional use in unusual
situations. They are here for reasons of pragmatism, not because we
expect them to be used routinely or recommend their use.
2021-03-16 16:26:22 -07:00
Paddy 0b5c4a6a2c
Accept TF_LOG=json to enable TRACE logs in JSON format
This is not currently a supported interface, but we plan to release
tool(s) that consume parts of it that are more dependable later,
separately from Terraform CLI itself.
2021-03-16 14:59:15 -07:00
James Bardin acfa79b7b2
Merge pull request #28116 from hashicorp/jbardin/hcl-cty-update
update hcl and go-cty
2021-03-16 17:14:47 -04:00
James Bardin 523eddd760 update hcl and go-cty 2021-03-16 16:50:30 -04:00
Pam Selle 81e8167a4c Update documentation for provider_sensitive_attrs experiment
Update documentation to reference that this experiment is now
default in 0.15+
2021-03-16 13:31:43 -04:00
James Bardin 1338502c7b
Merge pull request #26924 from remilapeyre/concurrent-locks-pg
Use a global sequence to create the IDs for each workspace
2021-03-16 11:28:04 -04:00
Alisdair McDiarmid 46a29b13ed cli: Add format version to validate -json output
In line with the other complex JSON output formats for plan and provider
schema, here we add an explicit `format_version` field to the JSON
output of terraform validate.
2021-03-16 09:46:36 -04:00
Alisdair McDiarmid d4e7a74f7e
Merge pull request #28093 from hashicorp/alisdair/fix-new-remote-workspace-state-migration
backend/remote: Fix new workspace state migration
2021-03-16 09:32:49 -04:00
Jason Smith 188ea61a12 Fix duplicate default state
The default state is already explicitly added to the result slice. Added
a guard to prevent it being added a second time.

Fixes https://github.com/hashicorp/terraform/issues/28098
2021-03-15 16:33:31 -05:00
Jason Smith 3e8ebd6f40 Fix type conversion panic
etcdv3 acceptance tests fail due to attempting to pass slices of strings
for the endpoints config to HCL2ValueFromConfigValue() which does not
handle that type.

Not a pretty solution but a helper function that converts the endpoints to a slice of
empty interfaces satisfies the requirements of the
HCL2ValueFromConfigValue function.

fixes https://github.com/hashicorp/terraform/issues/28096
2021-03-15 16:09:44 -05:00
Alisdair McDiarmid 4b159416ff backend/remote: Fix new workspace state migration
When migrating state to a new workspace, the version check would error
due to a 404 error on fetching the workspace record. This would result
in failed state migration.

Instead we should look specifically for a 404 error, and allow migration
to continue. If we're just about to create the workspace, there can't be
a version incompatibility problem.
2021-03-15 15:48:14 -04:00
Pam Selle 34536daff9
Merge pull request #28036 from hashicorp/pselle/provider_sensitivity_non-experiment
Make provider sensitivity default behavior
2021-03-15 10:23:04 -04:00
Kristin Laemmert fac60ab124
Add formatting for attributes with nested types (#28069)
* format/diff: extract attributes-writing logic to a function

This is a stepping-stone commit (for easier reviewability, and to prove that tests did not change) as part of writing a NestedType-specific diff printer.

* command/format: add support for formatting attributes with NestedTypes

This commit adds custom formatting for NestedType attributes. THe logic was mostly copied from the block diff printer, with minor tweaks here and there. I used the (excellent) existing test coverage and added a NestedType attribute to every test.

Since the (nested-block specific) test schemas were nearly identical, I added a function that returns the schema with the requested NestingMode.
2021-03-15 09:31:23 -04:00
Alisdair McDiarmid 783936f4c7
Merge pull request #28067 from hashicorp/alisdair/defaults-fix-null-structural-types
functions: Fix defaults for null objects/tuples
2021-03-15 08:03:08 -04:00
Alisdair McDiarmid 8a77f3f25f
Merge pull request #28057 from hashicorp/alisdair/json-diagnostics
cli: Add comprehensive JSON diagnostic structure
2021-03-15 08:02:49 -04:00
Alisdair McDiarmid 7f97bd4489 functions: Fix missing defaults for objects/tuples
If no default is specified for a nested optional structural typed
attribute, the defaults function should just pass through its input.
Before this commit the function assumed that the fallback value was
always of the correct type, which would panic.
2021-03-12 18:14:14 -05:00
James Bardin 0750a16cce
Merge pull request #28072 from hashicorp/jbardin/provider-meta-interpolation
validate provider_meta contains no interpolations
2021-03-12 17:12:58 -05:00
Alisdair McDiarmid 2a85f0483f website: Update validate -json diags documentation
Updated to include details about the new "snippet" object in JSON
diagnostics.
2021-03-12 14:25:11 -05:00
Alisdair McDiarmid c92826c14d cli: Use standard JSON diagnostics for validate
Now that we have a comprehensive JSON diagnostic structure, we can use
it in the `validate -json` output instead of the inline version. Note
that this changes the output of `validate -json` in two ways:

1. We fix some off-by-one errors caused by zero-width highlight ranges.
   This aligns the JSON diagnostic output with the text output seen by
   most Terraform users, so I consider this a bug fix.
2. We add the `snippet` field to the JSON diagnostics where available.
   This is purely additive and is permitted under our JSON format
   stability guarantees.
2021-03-12 13:12:37 -05:00
Alisdair McDiarmid d9d88b9243 cli: Add comprehensive JSON diagnostic structure
This commit adds a comprehensive JSON format for diagnostics, which
ensures that all current diagnostic output can be semantically
represented in a machine-readable format. The diagnostic formatter
interface remains unchanged, but it first transforms its input via the
JSON format to ensure that there is only one code path for creating the
diagnostic data.

The JSON diagnostic renderer extracts the non-presentational logic from
the format package, and returns a structure which can either be
marshaled into JSON or rendered as text. The resulting text diagnostic
output is unchanged for all cases covered by unit tests and my own
manual testing.

Included in this commit are a number of golden reference files for the
marshaled JSON output of a diagnostic. This format should change rarely
if at all, and these are in place to ensure that any changes to the
format are intentional and considered.
2021-03-12 13:12:37 -05:00
Martin Atkins dc7f2b7314 website: docs for the terraform validate JSON output 2021-03-12 09:39:56 -08:00
James Bardin 759b76436a validate provider_meta contains no interpolations
The provider_meta specification does not allow interpolation, but we
were not preventing it in the configuration.
2021-03-12 12:10:25 -05:00
Kristin Laemmert b26ae9cf48
add support for attributes with nested types in providers schema (#28055)
This PR extends jsonprovider to support attributes with NestedTypes and extends test coverage in jsonprovider and the providers schemas tests. I've also cleaned up some comments and extracted the logic to parse the nesting mode so it can be used in both marshalling blocks and attributes.
2021-03-12 08:28:22 -05:00
Alisdair McDiarmid 0bbe583eb8 functions: Fix defaults for null objects/tuples
When using defaults with a value which contains null objects or tuples,
we cannot continue to traverse the value and apply defaults. Instead,
when we find an attribute which is null, we return early and stop
processing this branch.
2021-03-12 08:23:41 -05:00
Nick Fagerlund 42c6c5dd6c
Merge pull request #28035 from antoineco/hcl-spec-link
docs: Update link to HCL native syntax spec
2021-03-11 11:03:01 -08:00
Pam Selle 242f319638
Give suggestions & remind users to use required_providers when provider not in registry (#28014)
* Add helper suggestion when failed registry err

When someone has a failed registry error on init, remind them that
they should have required_providers in every module

* Give suggestion for a provider based on reqs

Suggest another provider on a registry error, from the list of
requirements we have on init. This skips the legacy lookup
process if there is a similar provider existing in requirements.
2021-03-11 08:54:18 -05:00
James Bardin c5428959b5
Merge pull request #28043 from hashicorp/jbardin/missing-deposed-state
do not panic if there is no deposed state
2021-03-11 08:52:43 -05:00
James Bardin 1d4e1ed2b7 do not panic if there is no deposed state
NodeDestroyDeposedResourceInstanceObject should not panic if the deposed
state no longer exists.
2021-03-10 16:48:30 -05:00
Jason Smith 024035114a Fix broken link error
Link to https://consul.io/ is not actually broken but returns a 307
redirect to https://www.consul.io/ which the broken link test counts as a failure.
2021-03-10 15:38:22 -06:00
Jason Smith 8663b2100a Indicate etcdv3 support for multiple workspaces 2021-03-10 15:36:37 -06:00
Pam Selle 6ff1d2932e Conclude provider sensitivity experiment
Conclude the provider sensitivity experiment and make this
a default behavior.
2021-03-10 12:10:26 -05:00
Antoine Cotten 3442873cdb
docs: Update link to HCL native syntax spec 2021-03-10 17:47:21 +01:00
James Bardin e5538693ce
Merge pull request #28017 from hashicorp/jbardin/data-destroy-deps-2
destroying data source does not require a provider
2021-03-09 12:59:25 -05:00
Masayuki Morita 31a5aa1878
command/init: Add a new flag `-lockfile=readonly` (#27630)
Fixes #27506

Add a new flag `-lockfile=readonly` to `terraform init`.
It would be useful to allow us to suppress dependency lockfile changes
explicitly.

The type of the `-lockfile` flag is string rather than bool, leaving
room for future extensions to other behavior variants.

The readonly mode suppresses lockfile changes, but should verify
checksums against the information already recorded. It should conflict
with the `-upgrade` flag.

Note: In the original use-case described in #27506, I would like to
suppress adding zh hashes, but a test code here suppresses adding h1
hashes because it's easy for testing.

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2021-03-09 11:12:00 -05:00
James Bardin e6ab48addf test data source destroy with no provider 2021-03-08 21:43:47 -05:00
James Bardin 58b085f0dc prune unused providers from the graph, again
The provider transformers remove extra provider nodes when they are
initially setup, but it may turn out that they are not used later on.

The pruneUnusedNodesTransformer takes care of removing unused expansion
nodes, which originally required a provider, and hence may cause some
provider nodes to no longer be needed. We can also detect these and
remove them during the pruneUnusedNodesTransformer process.
2021-03-08 21:43:47 -05:00
James Bardin 7674e19d4e data source destroy nodes do not need a provider
Removing a data source is a state-only operation, and the node itself
does not require a provider.
2021-03-08 15:40:31 -05:00
James Bardin 3c6b2a8780 rename attachDataResourceDependsOnTransformer
Clarify the use of this transformer, interface and method which now does
not apply to anything but `depends_on` for data sources,
2021-03-08 15:39:09 -05:00
Alisdair McDiarmid 98899df514
Merge pull request #27979 from hashicorp/alisdair/defaults-fixes
functions: Fix panics in defaults
2021-03-05 14:39:50 -05:00
Alisdair McDiarmid ac8c6c1aa7
Merge pull request #27998 from hashicorp/alisdair/faster-addr-equals
core: Reduce string allocations for addrs Equal
2021-03-05 14:39:39 -05:00
Alisdair McDiarmid a43b035a51 core: Reduce string allocations for addrs Equal
Generating strings and comparing them to implement Equal is a quick and
easy solution. Unfortunately when this code is in the hot path, it
becomes very expensive, so this commit changes some of those instances
to compare the values directly.

Combined with using addr.Equal instead of checking for string equality,
this makes Terraform dramatically faster for some operations, such as
generating large JSON plans.
2021-03-05 13:23:32 -05:00
Alisdair McDiarmid 3d8b43dfe7
Merge pull request #27944 from hashicorp/alisdair/fix-missing-source-in-diags
views: Fix missing source in diagnostic output
2021-03-04 12:28:42 -05:00
Alisdair McDiarmid 66f8d1c1c2 functions: Fix defaults mismatched types fallback
We allow primitive fallback values which have mismatched types, but only
if there is a conversion to the target type. Previously we would allow
unsafe conversions (e.g. string to bool), but later had no capacity to
return an error if the conversion failed due to the value of the
fallback being unable to convert to the target type.

This commit makes the more conservative requirement that default
fallback values must have a safe conversion.
2021-03-04 10:43:09 -05:00
Alisdair McDiarmid 178a9b32d7 functions: Fix defaults null collections panic
When applying default values to collection types, null collections in
the input should result in empty collections in the output.
2021-03-04 10:13:41 -05:00
Omar Ismail e9c7f37b8c
Enable --auto-approve for Policy checks on Remote Backend (Terraform Cloud) (#27804)
* Fix auto-approve for soft-policy
* Update error handling
* update testing string for consistency
2021-03-01 08:54:30 -05:00
Alisdair McDiarmid 86495f93cb views: Fix missing source in diagnostic output
The previous implementation of views was copying and embedding the base
View struct in each individual view. While this allowed for easy access
to the interface of that struct (both in the view and externally), it more
importantly completely broke the ability of the diagnostic printer to
output source code snippets.

This is because the `configSources` field on the base view is lazily set
after the config loader is initialized. In the commands ported to use
views, this happens after the base View struct is copied, so we are
updating the wrong copy of the struct.

This commit fixes this with a simple mechanical refactor: keep a pointer
to the base View struct instead, and update all of the individual views
to explicitly refer to that struct to access its fields and methods.

This is not a particularly satisfying solution, but I can't find
anything clearly better. It might be worth exploring the alternative
approach in the view for the new test command, which explicitly pulls
its dependencies out of the base view, rather than retaining a full
reference. Maybe there's a third way which is better still.
2021-02-26 16:43:03 -05:00
Martin Atkins b5adc33075 configs: Accept and minimally validate a "language" argument
We expect that in order to continue to evolve the language without
breaking existing modules we will at some point need to have a way to mark
when a particular module is expecting a newer interpretation of the
language.

Although it's too early to do any deep preparation for that, this commit
aims to proactively reserve an argument named "language" inside
"terraform" blocks, which currently only accepts the keyword TF2021 that
is intended to represent "the edition of the Terraform language as defined
in 2021".

That argument also defaults to TF2021 if not set, so in practice there's
no real reason to set this today, but this minimal validation today is
intended to give better feedback to users of older Terraform versions in
the event that we introduce a new language edition later and they try to
use an module incompatible with their Terraform version.
2021-02-26 11:43:32 -08:00
Martin Atkins 54cc4dadf6 configs: A better error message for custom variable validation
Our previous message conflated the requirement for a full sentence with
the suggestion to write in a style similar to Terraform's built-in error
messages, which created a sense that the system would actively reject an
error message written in another language.

There's no intent here to block writing error messages in other languages,
but there is a practical consideration that Terraform's UI output is
currently not localized and so consistency with Terraform's other output,
if that's important to a module author, will typically mean writing the
error message in English.
2021-02-26 09:36:42 -08:00
Aaron Lane f172585eaa Fix Kitchen-Terraform language in test experiment
This isn't incredibly important, but Kitchen-Terraform is written in and requires tests to be written in Ruby.
2021-02-25 14:12:00 -08:00
Alisdair McDiarmid 37fff9336a
Merge pull request #27929 from hashicorp/alisdair/command-views-cleanup
Some command views cleanup
2021-02-25 16:26:08 -05:00