Commit Graph

27183 Commits

Author SHA1 Message Date
Alisdair McDiarmid 39bc5e825b terraform: Check for sensitive values in outputs
Sensitive values may not be used in outputs which are not also marked
as sensitive. This includes values nested within complex structures.

Note that sensitive values are unmarked before writing to state. This
means that sensitive values used in module outputs will have the
sensitive mark removed. At the moment, we have not implemented
sensitivity propagation from module outputs back to value marks.

This commit also reworks the tests for NodeApplyableOutput to cover
more existing behaviour, as well as this change.
2020-09-25 16:04:06 -04:00
Daniel Dreier 6211020453
Merge pull request #26247 from hashicorp/danieldreier/bugtriage
Add bug triage guide
2020-09-25 15:59:05 -04:00
Daniel Dreier 8af2120e1b Add bug triage guide
Add a written bug triage process and link to it in README.md

Bug process

Remove goals, edit for brevity, and move how to write a good issue report to bug report template

link HashiCorp GPG key in bug report template

add summary links for triage process
2020-09-25 15:56:58 -04:00
James Bardin ba7a57d3c5
Merge pull request #26375 from hashicorp/jbardin/data-force-plan-read
data source depends_on
2020-09-25 13:57:06 -04:00
James Bardin ea9096fb21 data source depends_on
A data source referencing another data source through depends_on should
not be forced to defer until apply. Data sources have no side effects,
so nothing should need to be applied. If the dependency has a
planned change due to a managed resource, the original data source will
also encounter that further down the list of dependencies.

This prevents a data source being read during plan for any reason from
causing other data sources to be deferred until apply. It does not
change the behavior noticeably in 0.14, but because 0.13 still had
separate refresh and plan phases which could read the data source, the
deferral could cause many things downstream to become unexpectedly
unknown until apply.
2020-09-25 13:46:47 -04:00
Pam Selle 40ea3f4cb8
Merge pull request #26373 from hashicorp/pselle/sensitive-vals-list
Support list diffs with sensitivity
2020-09-25 13:46:37 -04:00
Pam Selle 0b3c21a3eb Support lists of deeply marked values 2020-09-25 13:33:44 -04:00
Alisdair McDiarmid 1d9f2c5efc
Merge pull request #26378 from hashicorp/alisdair/complex-sensitive-values
Complex sensitive values
2020-09-25 11:59:43 -04:00
Pam Selle 715821a327
Merge pull request #26374 from hashicorp/pselle/sens-warning-color
Change sensitivity warning to be yellow only on 'Warning'
2020-09-25 11:42:36 -04:00
Alisdair McDiarmid e41b2ef2d4 terraform: Add test for complex sensitive values
When a sensitive variable has a complex type, any traversal of the
variable should still result in a sensitive value. This test uses a
sensitive `map(string)` and verifies that both plan and state output
include the appropriate sensitive marks for the resource attribute.
2020-09-25 11:37:45 -04:00
Alisdair McDiarmid 3f6b4cc62a go get github.com/hashicorp/hcl/v2@a0de289809fb 2020-09-25 11:30:27 -04:00
Alisdair McDiarmid 196c15a45d go get github.com/zclconf/go-cty@36785d4dc4ac 2020-09-25 10:27:53 -04:00
Pam Selle 634e83ab63 Change sensitivity warning to be yellow only on 'Warning' 2020-09-25 10:22:56 -04:00
Pam Selle 3dde9efc75 Support list diffs with sensitivity
Adds support for specialized diffs with lists
2020-09-25 10:18:33 -04:00
Kristin Laemmert 4bba9a70b3 terraform: refactor NodePlannableResource and NodeApplyableResource
NodePlannableResource and NodeApplyableResource EvalTree()s have been
replaced with Execute() nodes and straight-through code. Both called
EvalWriteResourceState and were the only functions to use it, so I chose
to replace EvalWriteResourceState entirely with straight-through code
(by copying the contents into the two locations).
2020-09-25 09:29:18 -04:00
Kristin Laemmert 90655b98b0 terraform: rename mustReourceAddr to mustConfigResourceAddr and add mustAbsResourceAddr
there are too many things that can be called resource addrs and it can
be hard to find the must* I'm looking for, so I renamed one and added
another.
2020-09-25 09:29:18 -04:00
Pam Selle f2f84003ee
Merge pull request #26367 from hashicorp/pselle/sensitive-diff-format
Warnings and specialized diffs when switching between sensitive values
2020-09-24 17:45:50 -04:00
Martin Atkins 532d68d2ed
CHANGELOG: Differences that come from our upgrade to Go 1.15 2020-09-24 14:34:52 -07:00
Martin Atkins ef64df950c getproviders: Prepare for having multiple valid hashes per package
As we continue iterating towards saving valid hashes for a package in a
depsfile lock file after installation and verifying them on future
installation, this prepares getproviders for the possibility of having
multiple valid hashes per package.

This will arise in future commits for two reasons:
- We will need to support both the legacy "zip hash" hashing scheme and
  the new-style content-based hashing scheme because currently the
  registry protocol is only able to produce the legacy scheme, but our
  other installation sources prefer the content-based scheme. Therefore
  packages will typically have a mixture of hashes of both types.
- Installing from an upstream registry will save the hashes for the
  packages across all supported platforms, rather than just the current
  platform, and we'll consider all of those valid for future installation
  if we see both successful matching of the current platform checksum and
  a signature verification for the checksums file as a whole.

This also includes some more preparation for the second case above in that
signatureAuthentication now supports AcceptableHashes and returns all of
the zip-based hashes it can find in the checksums file. This is a bit of
an abstraction leak because previously that authenticator considered its
"document" to just be opaque bytes, but we want to make sure that we can
only end up trusting _all_ of the hashes if we've verified that the
document is signed. Hopefully we'll make this better in a future commit
with some refactoring, but that's deferred for now in order to minimize
disruption to existing codepaths while we work towards a provider locking
MVP.
2020-09-24 14:01:54 -07:00
Martin Atkins 6694cfaa0e getproviders: Add a real type Hash for package hashes
The logic for what constitutes a valid hash and how different hash schemes
are represented was starting to get sprawled over many different files and
packages.

Consistently with other cases where we've used named types to gather the
definition of a particular string into a single place and have the Go
compiler help us use it properly, this introduces both getproviders.Hash
representing a hash value and getproviders.HashScheme representing the
idea of a particular hash scheme.

Most of this changeset is updating existing uses of primitive strings to
uses of getproviders.Hash. The new type definitions are in
internal/getproviders/hash.go.
2020-09-24 14:01:54 -07:00
Martin Atkins 264a3cf031 depsfile: Flatten the "hashes" locks to a single set of strings
Although origin registries return specific [filename, hash] pairs, our
various different installation methods can't produce a structured mapping
from platform to hash without breaking changes.

Therefore, as a compromise, we'll continue to do platform-specific checks
against upstream data in the cases where that's possible (installation
from origin registry or network mirror) but we'll treat the lock file as
just a flat set of equally-valid hashes, at least one of which must match
after we've completed whatever checks we've made against the
upstream-provided checksums/signatures.

This includes only the minimal internal/getproviders updates required to
make this compile. A subsequent commit will update that package to
actually support the idea of verifying against multiple hashes.
2020-09-24 14:01:54 -07:00
Martin Atkins b2c0ccdf96 internal/getproviders: Allow PackageMeta to carry acceptable hashes
The "acceptable hashes" for a package is a set of hashes that the upstream
source considers to be good hashes for checking whether future installs
of the same provider version are considered to match this one.

Because the acceptable hashes are a package authentication concern and
they already need to be known (at least in part) to implement the
authenticators, here we add AcceptableHashes as an optional extra method
that an authenticator can implement.

Because these are hashes chosen by the upstream system, the caller must
make its own determination about their trustworthiness. The result of
authentication is likely to be an input to that, for example by
distrusting hashes produced by an authenticator that succeeds but doesn't
report having validated anything.
2020-09-24 14:01:54 -07:00
Martin Atkins e843097e52 internal/getproviders: Formalize the "ziphash" hashing scheme
This is the pre-existing hashing scheme that was initially built for
releases.hashicorp.com and then later reused for the provider registry
protocol, which takes a SHA256 hash of the official distribution .zip file
and formats it as lowercase hex.

This is a non-ideal hash scheme because it works only for
PackageLocalArchive locations, and so we can't verify package directories
on local disk against such hashes. However, the registry protocol is now
a compatibility constraint and so we're going to need to support this
hashing scheme for the foreseeable future.
2020-09-24 14:01:54 -07:00
Pam Selle 5b549224ae Refactor to call ContainsMarked less and use len() instead 2020-09-24 16:42:03 -04:00
Alisdair McDiarmid 0b632c872e
Update CHANGELOG.md 2020-09-24 15:54:50 -04:00
Alisdair McDiarmid 390a3c1102
Update CHANGELOG.md 2020-09-24 15:53:58 -04:00
Alisdair McDiarmid 60c469b4a5
Merge pull request #26345 from hashicorp/alisdair/taint-should-respect-required-version
command: Taint should respect required_version
2020-09-24 15:52:23 -04:00
Pam Selle 3c9fad0b0e Move plan action check into the sensitivity warning method 2020-09-24 13:49:34 -04:00
Pam Selle 531728f6e9 Sensitive diffs for primitive types
When showing primitive type diffs, hide possibly
sensitive values
2020-09-24 13:27:15 -04:00
Pam Selle 20921dbfb8 Add warning about sensitivity change
This commit adds a warning before displaying
a sensitive diff, and always obfuscates the old value (even
if it was not previously marked as sensitive)
2020-09-24 12:57:40 -04:00
Pam Selle 0a02e7040f
Store sensitive attribute paths in state (#26338)
* Add creation test and simplify in-place test

* Add deletion test

* Start adding marking from state

Start storing paths that should be marked
when pulled out of state. Implements deep
copy for attr paths. This commit also includes some
comment noise from investigations, and fixing the diff test

* Fix apply stripping marks

* Expand diff tests

* Basic apply test

* Update comments on equality checks to clarify current understanding

* Add JSON serialization for sensitive paths

We need to serialize a slice of cty.Path values to be used to re-mark
the sensitive values of a resource instance when loading the state file.
Paths consist of a list of steps, each of which may be either getting an
attribute value by name, or indexing into a collection by string or
number.

To serialize these without building a complex parser for a compact
string form, we render a nested array of small objects, like so:

[
  [
    { type: "get_attr", value: "foo" },
    { type: "index", value: { "type": "number", "value": 2 } }
  ]
]

The above example is equivalent to a path `foo[2]`.

* Format diffs with map types

Comparisons need unmarked values to operate on,
so create unmarked values for those operations. Additionally,
change diff to cover map types

* Remove debugging printing

* Fix bug with marking non-sensitive values

When pulling a sensitive value from state,
we were previously using those marks to remark
the planned new value, but that new value
might *not* be sensitive, so let's not do that

* Fix apply test

Apply was not passing the second state
through to the third pass at apply

* Consistency in checking for length of paths vs inspecting into value

* In apply, don't mark with before paths

* AttrPaths test coverage for DeepCopy

* Revert format changes

Reverts format changes in format/diff for this
branch so those changes can be discussed on a separate PR

* Refactor name of AttrPaths to AttrSensitivePaths

* Rename AttributePaths/attributePaths for naming consistency

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-09-24 12:40:17 -04:00
James Bardin a7c5a72c3d
Merge pull request #26358 from hashicorp/jbardin/no-vendor
Remove vendoring
2020-09-24 10:57:02 -04:00
James Bardin 15dd493139
Merge pull request #26357 from hashicorp/jbardin/go-version
Update go to 1.15
2020-09-24 10:09:45 -04:00
James Bardin 435d8bdbae
Merge pull request #26343 from hashicorp/jbardin/update-cbd-state
Update create_before_destroy in state during refresh
2020-09-24 10:03:18 -04:00
James Bardin a0cee10720 add Addr field for logging 2020-09-24 09:49:22 -04:00
James Bardin eb17d9799b refresh cbd test 2020-09-24 09:43:48 -04:00
James Bardin 27809871ca update create_before_destroy when refreshing
In order to save any changes to lifecycle options, we need to record
those changes during refresh, otherwise they would only be updated when
there is a change in the resource to be applied.
2020-09-24 09:43:45 -04:00
James Bardin 014bd30a67
Merge pull request #26353 from hashicorp/jbardin/refresh-false
Re-implement -refresh=false
2020-09-24 09:40:33 -04:00
James Bardin 37569f5cc3 insert PlanRefresh into the context 2020-09-24 09:34:49 -04:00
James Bardin b16c600edc verify skipRefresh during plan 2020-09-24 09:34:49 -04:00
James Bardin 84f7116ac8 thread skipContext through to the instance node 2020-09-24 09:34:49 -04:00
James Bardin eebb4dfcb2 add SkipRefresh to the terraform context 2020-09-24 09:34:49 -04:00
James Bardin c2566bff7b
Merge pull request #26351 from hashicorp/jbardin/dependencies
we no longer need EvalRefreshDependencies
2020-09-24 09:33:59 -04:00
James Bardin 53a8e321eb remove mod=vendor in statemgr test 2020-09-24 08:44:49 -04:00
James Bardin 2e5366ac69 remove -mod=vendor from command tests 2020-09-23 17:56:19 -04:00
James Bardin 50e547db4d remove vendoring from contribution guide 2020-09-23 17:06:59 -04:00
James Bardin 8bfbafa790 remove vendor from some tooling 2020-09-23 17:06:59 -04:00
James Bardin bf5fc8b913 go1.15 in circleci 2020-09-23 16:54:37 -04:00
James Bardin 35a5b40500 remove vendor folder
Go modules are well understood and supported now, and since our build
pipeline no longer uses the vendored packages, we can remove the extra
overhead of maintaining these files.
2020-09-23 16:44:35 -04:00
James Bardin 6b6499d61c update go.sum for go1.15 2020-09-23 16:43:43 -04:00