Commit Graph

28468 Commits

Author SHA1 Message Date
James Bardin 98d01615f6 unreachable 2020-10-14 14:09:17 -04:00
James Bardin c61a893590 unused tests
these are no longer relevant
2020-10-14 14:08:09 -04:00
James Bardin 5e9425b562 unreachable 2020-10-14 14:06:00 -04:00
James Bardin b8df47c9ac add struct field names 2020-10-14 14:05:41 -04:00
Alisdair McDiarmid 8bd940d024 go get github.com/mattn/go-colorable@v0.1.8 2020-10-14 13:46:54 -04:00
Alisdair McDiarmid b1500db6b9
Merge pull request #26585 from hashicorp/alisdair/update-hcl-cty
Update hcl and go-cty dependencies
2020-10-14 13:42:46 -04:00
James Bardin 657dd33008
Merge pull request #26557 from remilapeyre/skip-ddl-commands
Add skip_table_creation and skip_index_creation options to the pg backend
2020-10-14 13:36:26 -04:00
James Bardin 08abf5d561
Merge pull request #26577 from hashicorp/jbardin/decoder-spec
Memoize Block.DecoderSpec
2020-10-14 12:45:23 -04:00
James Bardin e27ecba6e4 extended cache comments 2020-10-14 12:45:06 -04:00
Petros Kolyvas dc48450e79
Provisioner contribution guide updates (#26538)
An update on the deprecated state of vendor provisioners for our contribution guide.
2020-10-14 12:30:40 -04:00
Martin Atkins 0009768c7f internal/depsfile: Update the dependency lock file atomically
In this case, "atomic" means that there will be no situation where the
file contains only part of the newContent data, and therefore other
software monitoring the file for changes (using a mechanism like inotify)
won't encounter a truncated file.

It does _not_ mean that there can't be existing filehandles open against
the old version of the file. On Windows systems the write will fail in
that case, but on Unix systems the write will typically succeed but leave
the existing filehandles still pointing at the old version of the file.
They'll need to reopen the file in order to see the new content.
2020-10-14 08:01:19 -07:00
Martin Atkins 55e6f64977 internal/depsfile: Factor out our atomic file replacement logic
This originated in the cliconfig code to write out credentials files. The
Windows implementation of this in particular was quite onerous to get
right because it needs a very specific sequence of operations to avoid
running into exclusive file locks, and so by factoring this out with
only cosmetic modification we can avoid repeating all of that engineering
effort for other atomic file writing use-cases.
2020-10-14 08:01:19 -07:00
Martin Atkins e70ab09bf1 command: new cache directory .terraform/providers for providers
Terraform v0.10 introduced .terraform/plugins as a cache directory for
automatically-installed plugins, Terraform v0.13 later reorganized the
directory structure inside but retained its purpose as a cache.

The local cache used to also serve as a record of specifically which
packages were selected in a particular working directory, with the intent
that a second run of "terraform init" would always select the same
packages again. That meant that in some sense it behaved a bit like a
local filesystem mirror directory, even though that wasn't its intended
purpose.

Due to some unfortunate miscommunications, somewhere a long the line we
published some documentation that _recommended_ using the cache directory
as if it were a filesystem mirror directory when working with Terraform
Cloud. That was really only working as an accident of implementation
details, and Terraform v0.14 is now going to break that because the source
of record for the currently-selected provider versions is now the
public-facing dependency lock file rather than the contents of an existing
local cache directory on disk.

After some consideration of how to move forward here, this commit
implements a compromise that tries to avoid silently doing anything
surprising while still giving useful guidance to folks who were previously
using the unsupported strategy. Specifically:

- The local cache directory will now be .terraform/providers rather than
  .terraform/plugins, because .terraform/plugins is effectively "poisoned"
  by the incorrect usage that we can't reliably distinguish from prior
  version correct usage.

- The .terraform/plugins directory is now the "legacy cache directory". It
  is intentionally _not_ now a filesystem mirror directory, because that
  would risk incorrectly interpreting providers automatically installed
  by Terraform v0.13 as if they were a local mirror, and thus upgrades
  and checksum fetches from the origin registry would be blocked.

- Because of the previous two points, someone who _was_ trying to use the
  legacy cache directory as a filesystem mirror would see installation
  fail for any providers they manually added to the legacy directory.

  To avoid leaving that user stumped as to what went wrong, there's a
  heuristic for the case where a non-official provider fails installation
  and yet we can see it in the legacy cache directory. If that heuristic
  matches then we'll produce a warning message hinting to move the
  provider under the terraform.d/plugins directory, which is a _correct_
  location for "bundled" provider plugins that belong only to a single
  configuration (as opposed to being installed globally on a system).

This does unfortunately mean that anyone who was following the
incorrectly-documented pattern will now encounter an error (and the
aforementioned warning hint) after upgrading to Terraform v0.14. This
seems like the safest compromise because Terraform can't automatically
infer the intent of files it finds in .terraform/plugins in order to
decide automatically how best to handle them.

The internals of the .terraform directory are always considered
implementation detail for a particular Terraform version and so switching
to a new directory for the _actual_ cache directory fits within our usual
set of guarantees, though it's definitely non-ideal in isolation but okay
when taken in the broader context of this problem, where the alternative
would be silent misbehavior when upgrading.
2020-10-14 07:53:41 -07:00
James Bardin bb76c3b50c
Update configs/configschema/decoder_spec.go
Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-10-14 10:33:44 -04:00
Alisdair McDiarmid 948ab5ebca go get github.com/zclconf/go-cty@e5225636c8c2 2020-10-14 10:17:22 -04:00
Alisdair McDiarmid a647b11af8 go get github.com/hashicorp/hcl/v2@v2.7.0 2020-10-14 10:16:42 -04:00
James Bardin dd8a8bdea1 add benchmark used for DecoderSpec
Not a great benchmark, but record it here for posterity. Practical
testing with the AWS provider gives us a 98% speedup for simple plans.
2020-10-14 09:19:26 -04:00
James Bardin d40e2fb8d1 cache DecoderSpec calls
DecoderSpec may be called many times, and deeply recursive calls are
expensive. Since we cannot synchronize the Blocks themselves due to them
being copied in parts of the code, we use a separate cache to store the
generated Specs.
2020-10-14 09:19:26 -04:00
James Bardin d3307f4864
Merge pull request #26584 from hashicorp/jbardin/var-tests
re-enable and fix module variable tests
2020-10-14 09:16:59 -04:00
James Bardin 2e35ac12f3
Merge pull request #26583 from hashicorp/jbardin/go-cmp
update go-cmp
2020-10-14 09:12:44 -04:00
James Bardin 073beb90a6 re-enable and fix module variable tests
A few tests were inadvertently renamed, causing them to be be skipped.
For some reason this is not caught by the `vet` pass that happens during
normal testing.
2020-10-14 09:10:37 -04:00
James Bardin e20a01a292 update go-cmp
Update go-cmp to prevent pointer arithmetic panics when using the race
detector.
2020-10-14 08:37:14 -04:00
Pam Selle 8f72f4f317
Merge pull request #21936 from tiny-dancer/patch-1
Terraform Plan CLI Vars Format
2020-10-13 16:18:39 -04:00
Pam Selle 98603a7c51
Merge pull request #20036 from scraly/patch-2
feat(environment variable): add TF_WORKSPACE information
2020-10-13 16:11:09 -04:00
Pam Selle 305c6fc029
Merge branch 'master' into patch-2 2020-10-13 16:07:28 -04:00
Pam Selle 31805ef04c
Merge pull request #26575 from hashicorp/pselle/chdir-fix
Fix un-saved error on chdir
2020-10-13 16:02:49 -04:00
Alisdair McDiarmid a275d40274
Merge pull request #26573 from hashicorp/alisdair/show-diffs-when-only-sensitivity-changes
command: Show diffs when only sensitivity changes
2020-10-13 14:56:51 -04:00
Pam Selle 328baaad84 Fix un-saved error on chdir 2020-10-13 14:22:25 -04:00
Alisdair McDiarmid c798dc98db command: Show diffs when only sensitivity changes
When an attribute's sensitivity changes, but its value remains the same,
we consider this an update operation for the plan. This commit updates
the diff renderer to match this, detecting and displaying the change in
sensitivity.

Previously, the renderer would detect no changes to the value of the
attribute, and consider it a no-op action. This resulted in suppression
of the attribute when the plan is in concise mode.

This is achieved with a new helper function, ctyEqualValueAndMarks. We
call this function whenever we want to check that two values are equal
in order to determine whether the action is update or no-op.
2020-10-13 13:55:16 -04:00
Sean Teeling 7d6ec431d2 test locks in non-default workspace 2020-10-13 08:18:54 -07:00
Pam Selle fcae49611c
Merge pull request #26555 from hashicorp/pselle/sensitive-var-value-compat
Avoid disclosing values in errors on marked vals
2020-10-13 10:51:25 -04:00
Kristin Laemmert 57fd4c34d1 terraform: fix ProviderConfigTransformer
The ProviderConfigTransformer was using only the provider FQN to attach
a provider configuration to the provider, but what it needs to do is
find the local name for the given provider FQN (which may not match the
type name) and use that when searching for matching provider
configuration.

Fixes #26556

This will also be backported to the v0.13 branch.
2020-10-13 10:07:25 -04:00
Kristin Laemmert 2a478ed905 this is still used, let's leave it in place for now 2020-10-13 10:03:24 -04:00
Sean Teeling e1f1b84d67 Refresh state outside of grabbing the lock; only grab the lock on provisioning if the state file doesn't exist; this is similar to the GCS backend 2020-10-12 21:45:25 -07:00
James Bardin 5677978eb0
Merge pull request #26551 from hashicorp/jbardin/render-output-changes
Render output changes based on the plan
2020-10-12 19:05:10 -04:00
James Bardin 241765f0ab don't check outputs for legacy tests
The legacy tests never had to account for outputs in the plan. This path
is not used outside of old builtin test provider, so just work around
the output changes until we remove this completely.
2020-10-12 18:59:14 -04:00
James Bardin 5eca0788c6 rely solely on the plan changes for outputs
Now that outputs changes are tracked in full, we can remove the
comparisons with the prior state and use the planned changes directly.
2020-10-12 18:59:14 -04:00
Martin Atkins e1aff2bab0 website: First draft of v0.14 upgrade guide
The upgrade requirements for this release are considerably more modest
than for Terraform v0.13, so this time we just have some notes about a
few changes in behavior that may be impactful to some users.

This first pass is intended to be included as part of a forthcoming beta
testers' guide as we begin the v0.14 beta testing period. We will make
further changes to this upgrade guide based on feedback from those who
participate in the beta process.

Note that this upgrade guide is not intended as release marketing material
and so its presentation is focused on addressing concerns users might
encounter while upgrading. We'll share highlights from the release in
other contexts, such as the changelog and in the product blog.
2020-10-12 15:29:42 -07:00
James Bardin 03640057be
Merge pull request #26533 from hashicorp/jbardin/plan-output-changes
Use recorded changes for outputs and plan root output removals
2020-10-12 17:35:36 -04:00
James Bardin 28e4281674 handle sensitivity in the OutputChange
The state is not loaded here with any marks, so we cannot rely on marks
alone for equality comparison. Compare both the state and the
configuration sensitivity before creating the OutputChange.
2020-10-12 17:29:45 -04:00
James Bardin d2514a9abd update new outputs plan json 2020-10-12 17:29:45 -04:00
James Bardin d82778f4fc insert before values into the output changes
Lookup before values for output changes.
Use Update action when output has a non-null before value.
2020-10-12 17:29:45 -04:00
James Bardin 0f5bf21983 remove last use of the apply graph Destroy flag!
The apply graph builder no longer uses the destroy flag, which is not
always known since the destroy flag is not stored in the plan file.
2020-10-12 17:29:45 -04:00
James Bardin ff21cc3c8d remove the need for destroyRootOutputTransformer
Since root outputs can now use the planned changes, we can directly
insert the correct applyable or destroyable node into the graph during
plan and apply, and it will remove the outputs if they are being
destroyed.
2020-10-12 17:29:45 -04:00
Rémi Lapeyre 12a0a21c0b Add skip_table_creation and skip_index_creation options to the pg backend
Closes https://github.com/hashicorp/terraform/issues/25708
2020-10-12 22:47:19 +02:00
Pam Selle da4ddd0160 Avoid disclosing values in errors on marked vals
AssertObjectCompatible is a special case that will
expose Go string values of values unless otherwise
stopped. This adds that check.
2020-10-12 15:53:34 -04:00
Martin Atkins af20a769be
Update CHANGELOG.md 2020-10-12 10:21:49 -07:00
Martin Atkins 0bbbb9c64b configs: Experimental support for optional object type attributes
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.

In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.

Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
2020-10-12 10:12:28 -07:00
Pam Selle 18d59d768f
Update CHANGELOG.md 2020-10-12 10:08:55 -04:00
Pam Selle 3cba9b9968
Merge pull request #26543 from caarlos0/err
fix: update go-versions with improved error handling
2020-10-12 10:07:35 -04:00