Commit Graph

29351 Commits

Author SHA1 Message Date
Martin Atkins 74761b2f8b getmodules: Use go-getter v1.5.10 and return to upstream GitGetter
There was an unintended regression in go-getter v1.5.9's GitGetter which
caused us to temporarily fork that particular getter into Terraform to
expedite a fix. However, upstream v1.5.10 now includes a
functionally-equivalent fix and so we can heal that fork by upgrading.

We'd also neglected to update the Module Sources docs when upgrading to
go-getter v1.5.9 originally and so we were missing documentation about the
new "depth" argument to enable shadow cloning, which I've added
retroactively here along with documenting its restriction of only
supporting named refs.

This new go-getter release also introduces a new credentials-passing
method for the Google Cloud Storage getter, and so we must incorporate
that into the Terraform-level documentation about module sources.
2022-01-03 11:44:16 -08:00
James Bardin 66b4d155b1
Merge pull request #30232 from hashicorp/jbardin/module-move-re-index
Handle move blocks within a module which is changing the index
2021-12-22 16:27:30 -05:00
James Bardin 75ef61c783 check for nested module index changes
Changing only the index on a nested module will cause all nested moves
to create cycles, since their full addresses will match both the From
and To addresses. When building the dependency graph, check if the
parent is only changing the index of the containing module, and prevent
the backwards edge for the move.
2021-12-22 16:15:04 -05:00
Martin Atkins e35c25da44 website: Try function documentation "provably" vs "probably" typo
This paragraph is trying to say that try only works for dynamic errors and
not for errors that are _not_ based on dynamic decision-making in
expressions.

I'm not sure if this typo was always here or if it was mistakenly "corrected"
at some point, but either way the word "probably" changes the meaning
of this sentence entirely, making it seem like Terraform is hedging
the likelihood of a problem rather than checking exactly for one.
2021-12-22 12:10:07 -08:00
Barrett Clark 296acdd961
Merge pull request #30020 from hashicorp/barrettclark/cloud-e2e-parallel
Cloud: Add parallelism back into the tests
2021-12-22 12:14:06 -06:00
Alisdair McDiarmid f772cb085e
Merge pull request #30233 from hashicorp/alisdair/move-nested-modules
refactoring: Move nested modules
2021-12-22 10:22:21 -05:00
James Bardin a72d2d408d
Merge pull request #30095 from hashicorp/jbardin/invalid-provider-name
skip provider resolution when there are errors
2021-12-21 16:58:48 -05:00
James Bardin e761117562 find implied moves in nested modules
Implied moves in nested modules were being skipped
2021-12-21 16:49:25 -05:00
James Bardin 346418e31f IsModuleMoveReIndex
Add a method for checking if the From and To addresses in a move
statement are only changing the indexes of modules relative to the
statement module.

This is needed because move statement nested within the module will be
able to match against both the From and To addresses, causing cycles in
the order of move operations.
2021-12-21 16:49:25 -05:00
Alisdair McDiarmid d7ef123c12 refactoring: Move nested modules
When applying module `moved` statements by iterating through modules in
state, we previously required an exact match from the `moved`
statement's `from` field and the module address. This permitted moving
resources directly inside a module, but did not recur into module calls
within those moved modules.

This commit moves that exact match requirement so that it only applies
to `moved` statements targeting resources. In turn this allows nested
modules to be moved.
2021-12-21 16:25:06 -05:00
Laura Pacilio cd7277128f
Merge pull request #30223 from hashicorp/replace-flag-updates
Update taint command page to make alternative clearer
2021-12-21 10:42:46 -05:00
Barrett Clark d196d2870a Refactor cloud table test runs
As the cloud e2e tests evolved some common patters became apparent. This
standardizes and consolidates the patterns into a common test runner
that takes the table tests and runs them in parallel. Some tests also
needed to be converted to utilize table tests.
2021-12-20 16:36:06 -06:00
Laura Pacilio 2a530639f8 Tweaks 2021-12-20 16:01:03 -05:00
Laura Pacilio 410e45673d Update taint command page to make alternative clearer 2021-12-20 15:43:31 -05:00
Dylan Staley 8b6522169f
Merge pull request #30191 from hashicorp/ds.make-website
Update make website workflow
2021-12-20 11:10:59 -08:00
Martin Atkins 23395a1022 providercache: Discard lock entries for unused providers
Previously we would only ever add new lock entries or update existing
ones. However, it's possible that over time a module may _cease_ using
a particular provider, at which point we ought to remove it from the lock
file so that operations won't fail when seeing that the provider cache
directory is inconsistent with the lock file.

Now the provider installer (EnsureProviderVersions) will remove any lock
file entries that relate to providers not included in the given
requirements, which therefore makes the resulting lock file properly match
the set of packages the installer wrote into the cache.

This does potentially mean that someone could inadvertently defeat the
lock by removing a provider dependency, running "terraform init", then
undoing that removal, and finally running "terraform init" again. However,
that seems relatively unlikely compared to the likelihood of removing
a provider and keeping it removed, and in the event it _did_ happen the
changes to the lock entry for that provider would be visible in the diff
of the provider lock file as usual, and so could be noticed in code
review just as for any other change to dependencies.
2021-12-17 15:30:21 -08:00
James Bardin 2c8edfb259
Merge pull request #30199 from hashicorp/jbardin/apply-failure-diags
Apply graph failure handling
2021-12-17 14:08:02 -05:00
James Bardin 8c4031ef15 don't persist a nil state from Apply
Apply should not return a nil state to be persisted.
2021-12-17 14:00:59 -05:00
James Bardin 58e001c22a return graph errors from Context.Apply
errors from building during apply were lost
2021-12-17 14:00:57 -05:00
James Bardin f83ed441bb
Merge pull request #30189 from hashicorp/jbardin/validate-moves
Ignore unexpanded paths when validating move statements.
2021-12-17 13:57:24 -05:00
James Bardin 7e1646919b
Merge pull request #30185 from hashicorp/jbardin/panic-cleanup
Cleanup panic output
2021-12-17 13:56:31 -05:00
James Bardin b213386a73 InstancesForModule should not panic
instances.Set is only used after all instances have been processes, so
it should therefor only handle known instances and not panic when given
an address that traverses an unexpanded module.
2021-12-17 13:31:41 -05:00
James Bardin 371660ab8f cleanup panic output 2021-12-17 11:57:52 -05:00
Dylan Staley a086f0c1a5 update make website workflow 2021-12-16 16:10:17 -08:00
James Bardin a5017bff2f failing test moved with target 2021-12-16 18:20:49 -05:00
Barrett Clark c647b41d65 Add parallelism back into the tests
Running tests in parallel can help speed up overall test execution. Go
blocks parent tests while child tests run, so it does not fully fan out
as you might expect. It is noticably faster, though. Running 4 or more
concurrent processes knocks over a minute off the total execution time.
2021-12-15 11:37:49 -06:00
James Bardin e22ab70e03
Merge pull request #30171 from hashicorp/jbardin/revert-validate-for-each
use `cty.DynamicVal` for expanded resources during validation
2021-12-15 08:57:43 -05:00
James Bardin 645fcc5f12 test for lookup regression during validation 2021-12-15 08:43:37 -05:00
Dylan Staley 58e4676aea
Merge pull request #30173 from hashicorp/ds.mdx-migration-main
Migrate docs to MDX (main)
2021-12-14 18:58:19 -08:00
Dylan Staley 21cbb5b392 migrate docs to mdx 2021-12-14 18:41:17 -08:00
James Bardin 71b9682e8c ensure there is always a valid return value 2021-12-14 18:02:57 -05:00
James Bardin d469e86331 revert 6b8b0617
Revert the evaluation change from #29862.
While returning a dynamic value for all expanded resources during
validation is not optimal, trying to work around this using unknown maps
and lists is causing other undesirable behaviors during evaluation.
2021-12-14 17:58:10 -05:00
Martin Atkins c4d46e7c6b getmodules: Re-allow git:: source with ref=COMMIT_ID
Earlier versions of this code allowed "ref" to take any value that would
be accepted by "git checkout" as a valid target of a symbolic ref. We
inadvertently accepted a breaking change to upstream go-getter that broke
that as part of introducing a shallow clone optimization, because shallow
clone requires selecting a single branch.

To restore the previous capabilities while retaining the "depth" argument,
here we accept a compromise where "ref" has the stronger requirement of
being a valid named ref in the remote repository if and only if "depth"
is set to a value greater than zero. If depth isn't set or is less than
one, we will do the old behavior of just cloning all of the refs in the
remote repository in full and then switching to refer to the selected
branch, tag, or naked commit ID as a separate step.

This includes a heuristic to generate an additional error message hint if
we get an error from "git clone" and it looks like the user might've been
trying to use "depth" and "ref=COMMIT" together. We can't recognize that
error accurately because it's only reported as human-oriented git command
output, but this heuristic should hopefully minimize situations where we
show it inappropriately.

For now this is a change in the Terraform repository directly, so that we
can expedite the fix to an already-reported regression. After this is
released I tend to also submit a similar set of changes to upstream
go-getter, at which point we can revert Terraform to using the upstream
getter.GitGetter instead of our own local fork.
2021-12-14 11:24:23 -08:00
Martin Atkins b0ff17ef2a getmodules: Inline our own fork of getter.GitGetter
This is a pragmatic temporary solution to allow us to more quickly resolve
an upstream regression in go-getter locally within Terraform, so that the
work to upstream it for other callers can happen asynchronously and with
less time pressure.

This commit doesn't yet include any changes to address the bug, and
instead aims to be functionally equivalent to getter.GitGetter. A
subsequent commit will then address the regression, so that the diff of
that commit will be easier to apply later to the upstream to get the same
effect there.
2021-12-14 11:24:23 -08:00
Chris Arcand 8b8fe2771f
Merge pull request #30142 from hashicorp/chrisarcand/remote-backend-no-workspaces-regression
command/meta_backend: Allow the remote backend to have no workspaces [again]
2021-12-14 09:58:50 -06:00
Chris Arcand 98978b3853 command/meta_backend: Allow the remote backend to have no workspaces [again]
A regression introduced in d72a413ef8

The comment explains, but TLDR: The remote backend actually *depended*
on being able to write it's backend state even though an 'error'
occurred (no workspaces).
2021-12-14 09:50:42 -06:00
Alisdair McDiarmid aa59eb427a
Merge pull request #30151 from hashicorp/f-non-existing-module-instance-crash
core: Fix crash with orphaned module instance
2021-12-14 09:10:41 -05:00
Martin Atkins 096cddb4b7 command/format: Limitation of plans.ResourceInstanceDeleteBecauseNoModule
This is an explicit technical debt note that our plan renderer isn't able
to give a fully-specific hint in this particular case of deletion reason.

This reason code means that at least one of the module instance keys in
the resource's module path doesn't match an instance declared in the
configuration, but the plan data structure doesn't retain enough
information to know which is the first step in the path which refers to
a missing instance, and so we just always return the whole thing.

This would be confusing if we return module.foo[0].module.bar not being
in the configuration as a result of module.foo not using "count"; it would
be better to say "module.foo[0] is not in the configuration" instead.

It would be most ideal to handle all of the different situations that
ResourceInstanceDeleteBecauseWrongRepetition's rendering does, so that we
can go further and explain exactly _why_ that module instance isn't
declared anymore.

We can do neither of those things today because only the Terraform Core
"expander" component knows that information, and we've discarded that
by the time we get to rendering a plan. To fix this one day would require
preserving in the plan information about which module instances are
declared, as a separate sidecar data structure from which resource
instances we're taking actions on, and then using that to identify which
step in addr.Module here first selects an invalid instance.
2021-12-13 10:04:15 -05:00
Martin Atkins ec6fe93fa8 instances: Non-existing module instance has no resource instances
Previously we were treating it as a programming error to ask for the
instances of a resource inside an instance of a module that is declared
but whose declaration doesn't include the given instance key.

However, that's actually a valid situation which can arise if, for
example, the user has changed the repetition/expansion mode for an
existing module call and so now all of the resource instances addresses it
previously contained are "orphaned".

To represent that, we'll instead say that an invalid instance key of a
declared module behaves as if it contains no resource instances at all,
regardless of the configurations of any resources nested inside. This
then gives the result needed to successfully detect all of the former
resource instances as "orphaned" and plan to destroy them.

However, this then introduces a new case for
NodePlannableResourceInstanceOrphan.deleteActionReason to deal with: the
resource configuration still exists (because configuration isn't aware of
individual module/resource instances) but the module instance does not.
This actually allows us to resolve, at least partially, a previous missing
piece of explaining to the user why the resource instances are planned
for deletion in that case, finally allowing us to be explicit to the user
that it's because of the module instance being removed, which
internally we call plans.ResourceInstanceDeleteBecauseNoModule.

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2021-12-13 10:03:50 -05:00
Patrick Decat 6530055d18
website: Refactoring "Renaming a Module Call" should include a moved block 2021-12-09 14:54:19 -08:00
Laura Pacilio 7ae574ba6a
Merge pull request #30082 from c00k133/docs-vars-typo
Docs: Change misspelling of variable in documentation
2021-12-09 16:35:50 -05:00
Laura Pacilio cd8b458b9c
Merge pull request #30118 from marcusway/patch-1
Update docs to use `assume_role` block
2021-12-09 16:35:12 -05:00
Brandon Croft b27c28f341
Merge pull request #30126 from hashicorp/brandonc/go-tfe-0.21.0
Update go-tfe to v0.21.0
2021-12-09 13:58:17 -07:00
Brandon Croft ced4f5500c
update go-tfe to v0.21.0 2021-12-09 13:16:11 -07:00
Marcus Way b202eefc7b
Update docs to use `assume_role` block
I'm guessing this document is just out of date, but I got an error when configuring `assume_role` as an argument and had to add an `assume_role` block
2021-12-08 19:33:17 -05:00
Laura Pacilio 839458c192
Merge pull request #30116 from hashicorp/fix-links-release
Update broken links
2021-12-08 18:04:30 -05:00
Laura Pacilio 168c2885e3 Fix link text 2021-12-08 18:03:26 -05:00
Laura Pacilio 22d80df0ba Fix extra space in link 2021-12-08 18:01:48 -05:00
Laura Pacilio 2edd9f4e10 Update broken links 2021-12-08 17:53:18 -05:00
Martin Atkins 533a29edee website: Shorter heading for the nullable = false variable argument
This makes it match some incoming links we have elsewhere, but also it
makes the heading a bit more consice because "module" isn't really adding
anything here anyway: input variables are _always_ in modules.
2021-12-08 14:08:42 -08:00