Commit Graph

27185 Commits

Author SHA1 Message Date
James Bardin fd4f7eb0b9 remove prefixed io
The main process is now handling what output to print, so it doesn't do
any good to try and run it through prefixedio, which is only adding
extra coordination to echo the same data.
2020-10-21 17:29:07 -04:00
Jerry Chong 2f091836c9
Modified terraform get command (#26465)
-Added PATH
-Added -no-color option
2020-10-21 18:14:54 -03:00
Diod FR f0edb192b3
ADD CLI option position for force-unlock command (#26626)
* ADD CLI option position for force-unlock command

* Update force-unlock.html.markdown

Made a change to also include the missing [DIR]

Co-authored-by: Petros Kolyvas <petros@hashicorp.com>
2020-10-21 18:13:18 -03:00
Pam Selle c6be76e53c Return marked After values in GetResource
If a change exists for a resource instance,
the After value is returned, however, this value
will not have its marks as it as been encoded.
This Marks the return value so the marks follow
that resource reference.
2020-10-21 16:18:54 -04:00
Alisdair McDiarmid bc769aeb7c
Update CHANGELOG.md 2020-10-21 14:12:48 -04:00
Bishwa Shrestha c41336bc77
Exit with error if UI input scan fails (#26509) 2020-10-21 14:10:06 -04:00
Martin Atkins 70c52c778c
lang/funcs: textencodebase64 and textdecodebase64 2020-10-21 11:09:25 -07:00
Martin Atkins 1dc4950bfa lang/funcs: Rename the base64 character encoding functions
These were initially introduced as functions with "encode" and "decode"
prefixes, but that doesn't match with our existing convention of putting
the encoding format first so that the encode and decode functions will
group together in a alphabetically-ordered function list.

"text" is not really a defined serialization format, but it's a short word
that hopefully represents well enough what these functions are aiming to
encode and decode, while being consistent with existing functions like
jsonencode/jsondecode, yamlencode/yamldecode, etc.

The "base64" at the end here is less convincing because there is precedent
for that modifier to appear both at the beginning and the end in our
existing function names. I chose to put it at the end here because that
seems to be our emergent convention for situations where the base64
encoding is a sort of secondary modifier alongside the primary purpose
of the function, as we see with "filebase64". (base64gzip is an exception
here, but it seems outvoted by the others.)
2020-10-21 10:56:56 -07:00
James Bardin 1d9d82973b move panicwrap handler to logging package 2020-10-21 13:47:16 -04:00
r0bnet 877399c631 lang/funcs: Functions for encoding text in specific character encodings 2020-10-21 10:39:43 -07:00
Alisdair McDiarmid 7a31e56cb7
Merge pull request #26637 from hashicorp/alisdair/fix-locksfile-unconstrained-versions
internal: Fix lockfile constraint output for 1.2.*
2020-10-21 12:05:19 -04:00
Martin Atkins 1fe314b70b getproviders: Don't log if EvalSymlinks changes nothing
Previously this codepath was generating a confusing message in the absense
of any symlinks, because filepath.EvalSymlinks returns a successful result
if the target isn't a symlink.

Now we'll emit the log line only if filepath.EvalSymlinks returns a
result that's different in a way that isn't purely syntactic (which
filepath.Clean would "fix").

The new message is a little more generic because technically we've not
actually ensured that a difference here was caused by a symlink and so
we shouldn't over-promise and generate something potentially misleading.
2020-10-21 07:49:11 -07:00
Kristin Laemmert f211cf0c73 build: update comments in dockerfile 2020-10-21 08:49:36 -04:00
Kristin Laemmert 9d623290f4 command/e2etest: do not leave bad directories behind
If you run the e2etests locally and use a configured plugin_cache_dir,
the test will leave a bad directory behind in your cache dir that causes
later `init`s to fail. To circumvent this, pass an explicity-empty CLI
config file.

This is a nicety for local developers and not necessarily required, but
it happens to me often enough that I'd like to fix it. It's probably not
a *bad* idea to pass an explicit cli config to all e2etests, honestly,
but this is the only one that causes active problems so I limited this
PR to that one test.

Here's the error which occurs on subsequent `init` if this test is run on a
machine that uses a plugin cache dir:

2020/10/13 10:41:05 [TRACE] providercache.fillMetaCache: error while scanning directory /Users/mildwonkey/.terraform.d/plugin-cache: failed to read metadata about /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: stat /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: no such file or directory
2020-10-21 07:53:28 -04:00
Martin Atkins 2332a7ab47 replacefile: don't create the temporary file in TMPDIR
ioutil.TempFile has a special case where an empty string for its dir
argument is interpreted as a request to automatically look up the system
temporary directory, which is commonly /tmp .

We don't want that behavior here because we're specifically trying to
create the temporary file in the same directory as the file we're hoping
to replace. If the file gets created in /tmp then it might be on a
different device and thus the later atomic rename won't work.

Instead, we'll add our own special case to explicitly use "." when the
given filename is in the current working directory. That overrides the
special automatic behavior of ioutil.TempFile and thus forces the
behavior we need.

This hadn't previously mattered for earlier callers of this code because
they were creating files in subdirectories, but this codepath was failing
for the dependency lock file due to it always being created directly
in the current working directory.

Unfortunately since this is a picky implementation detail I couldn't find
a good way to write a unit test for it without considerable refactoring.
Instead, I verified manually that the temporary filename wasn't in /tmp on
my Linux system, and hope that the comment inline will explain this
situation well enough to avoid an accidental regression in future
maintenence.
2020-10-20 16:29:59 -07:00
Paul Thrasher 960042ad42
Merge pull request #26618 from hashicorp/pault/tfce-continue-on-error
Allow runs to continue on tfce error
2020-10-20 13:23:09 -07:00
Alisdair McDiarmid 9904667fc3
Merge pull request #26644 from hashicorp/alisdair/ignore-changes-sensitive
terraform: Fix sensitive values in ignore changes
2020-10-20 14:22:09 -04:00
Pam Selle e448a12c11
Merge pull request #26647 from hashicorp/pselle/codecov
[build] Only run codecov on master
2020-10-20 14:09:45 -04:00
Pam Selle df65de6945 Only run codecov on master 2020-10-20 13:54:48 -04:00
Pam Selle 7924117acc
Add note to upgrade guide about provider sensitivity (#26643)
* Add note to upgrade guide about provider sensitivity

Now that sensitivity follows attributes providers mark
as sensitive, add this note to the upgrade guide.

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-10-20 13:13:51 -04:00
Alisdair McDiarmid fb98fc98fa terraform: Fix sensitive values in ignore changes
Because ignore_changes configuration can refer to resource arguments
which are assigned sensitive values, we need to unmark the resource
object before processing.
2020-10-20 12:27:17 -04:00
Alisdair McDiarmid 9576a5b2d8 internal: Fix lockfile constraint output for 1.2.*
If a configuration requires a partial provider version (with some parts
unspecified), Terraform considers this as a constrained-to-zero version.
For example, a version constraint of 1.2 will result in an attempt to
install version 1.2.0, even if 1.2.1 is available.

When writing the dependency locks file, we previously would write 1.2.*,
as this is the in-memory representation of 1.2. This would then cause an
error on re-reading the locks file, as this is not a valid constraint
format.

Instead, we now explicitly convert the constraint to its zero-filled
representation before writing the locks file. This ensures that it
correctly round-trips.

Because this change is made in getproviders.VersionConstraintsString, it
also affects the output of the providers sub-command.
2020-10-20 10:14:03 -04:00
James Bardin d9ac57ffae
Merge pull request #26632 from hashicorp/jbardin/logging
Structured logging
2020-10-20 09:23:57 -04:00
James Bardin 18f676ea11
Merge pull request #26638 from hashicorp/jbardin/assert-compat-dynamic-blocks
Treat empty strings as null in NestingSet objs when looking for dynamic blocks in AssertObjectCompatible
2020-10-20 09:06:04 -04:00
James Bardin ef086399f9 compare empty strings as null in sets
The Legacy SDK cannot handle missing strings from objects in sets, and
will insert an empty string when planning the missing value. This
subverts the `couldHaveUnknownBlockPlaceholder` check, and causes
errors when `dynamic` is used with NestingSet blocks.

We don't have a separate codepath to handle the internals of
AssertObjectCompatible differently for the legacy SDK, but we can treat
empty strings as null strings within set objects to avoid the failed
assertions.
2020-10-19 18:07:45 -04:00
Pam Selle cdebf2820d
Merge pull request #26590 from hashicorp/pselle/sensitivity-providers
Mark attributes providers mark as sensitive
2020-10-19 16:13:39 -04:00
Pam Selle 394e60608c Allocate new copies of paths to avoid append drama
Create new copies of the Path to avoid possible append
related dramas. Also add a test to cover nested block
within blocks
2020-10-19 15:24:14 -04:00
James Bardin a3a20e0396 remove LevelFilter
We want to always be using the hclogger to filter whenever possible
2020-10-19 14:31:05 -04:00
James Bardin 0b31ffa587 use a single log writer
Use a single log writer instance for all std library logging.

Setup the std log writer in the logging package, and remove boilerplate
from test packages.
2020-10-19 14:29:54 -04:00
James Bardin abf6b9b378 get properly configured hcloggers
make sure plugins get hcloggers configured to match core
2020-10-19 14:29:54 -04:00
James Bardin 8a7838266e remove old plugin config
This is unused and leads to confusion
2020-10-19 14:29:54 -04:00
James Bardin 211edf5d75 use hclog as the default logger
Inject hclog as the default logger in the main binary.
2020-10-19 14:29:54 -04:00
James Bardin 776482cc59 update hclog 2020-10-19 14:29:52 -04:00
James Bardin 6ca477f042 move helper/logging to internal
remove a dead code file too
2020-10-19 14:27:53 -04:00
Alisdair McDiarmid 5e047b0a0b
Merge pull request #26611 from hashicorp/alisdair/sensitive-values-provisioners
Fixes for sensitive values used as input to provisioners
2020-10-19 13:39:18 -04:00
Alisdair McDiarmid 9d2994ab9b website: Sensitive values in provisioner config 2020-10-19 13:19:51 -04:00
Pam Selle b44dad3ce6
Merge pull request #26623 from hashicorp/pselle/cleanup
Comment fixing
2020-10-19 10:30:40 -04:00
Alisdair McDiarmid 24edf66c01
Update CHANGELOG.md 2020-10-19 10:06:07 -04:00
Alisdair McDiarmid e6d42ec4b3
Update CHANGELOG.md 2020-10-19 10:05:59 -04:00
Alisdair McDiarmid 4206b69060
Merge pull request #26588 from hashicorp/alisdair/upgrade-colorable
go get github.com/mattn/go-colorable@v0.1.8
2020-10-19 10:04:07 -04:00
James Bardin 46c0cedb20 handle empty json 2020-10-19 09:41:03 -04:00
James Bardin af9f985d66 Accept JSON encoded dynamic values from providers
Core was previously ignoring JSON-encoded dynamic values, but these are
technically supported, so we must either error or accept the value.
Since we already have the decoder for Json state, it's minimal effort to
support this on all plugin methods too.

This change also gives providers an easy way to implement the
UpgradeResourceState method. The obvious implementation of returning the same
JSON-encoded value has tripped up a few providers not using the legacy
SDK already, and we should have at least indicated that the value was
being lost.
2020-10-19 09:41:03 -04:00
James Bardin 488840171a
Merge pull request #26613 from hashicorp/jbardin/plugin-dynamic-json
Accept JSON encoded dynamic values from providers
2020-10-19 09:37:50 -04:00
James Bardin 353937411d handle empty json 2020-10-19 09:31:37 -04:00
Pam Selle 02c48f8071 Comment fixing 2020-10-18 13:00:09 -04:00
Paul Thrasher d380f5873c
Allow runs to continue on tfce error
A cost estimation error does not actually stop a run, so the run was continuing in the background after the cli exits, causing confusion. This change matches the UI behavior.
2020-10-16 17:27:04 -07:00
Martin Atkins 30204ecded command/cliconfig: Allow development overrides for providers
For normal provider installation we want to associate each provider with
a selected version number and find a suitable package for that version
that conforms to the official hashes for that release.

Those requirements are very onerous for a provider developer currently
testing a not-yet-released build, though. To allow for that case this new
CLI configuration feature allows overriding specific providers to refer
to give local filesystem directories.

Any provider overridden in this way is not subject to the usual
restrictions about selected versions or checksum conformance, and
activating an override won't cause any changes to the selections recorded
in the lock file because it's intended to be a temporary setting for one
developer only.

This is, in a sense, a spiritual successor of an old capability we had to
override specific plugins in the CLI configuration file. There were
some vestiges of that left in the main package and CLI config package
but nothing has actually been honoring them for several versions now and
so this commit removes them to avoid confusion with the new mechanism.
2020-10-16 14:31:15 -07:00
James Bardin 98dc399062
Merge pull request #26606 from hashicorp/jbardin/proposed-set-comparison
More precise handling of lists and maps within sets for ProposedNewObject
2020-10-16 16:21:12 -04:00
James Bardin e3804810a9 Accept JSON encoded dynamic values from providers
Core was previously ignoring JSON-encoded dynamic values, but these are
technically supported, so we must either error or accept the value.
Since we already have the decoder for Json state, it's minimal effort to
support this on all plugin methods too.

This change also gives providers an easy way to implement the
UpgradeResourceState method. The obvious implementation of returning the same
JSON-encoded value has tripped up a few providers not using the legacy
SDK already, and we should have at least indicated that the value was
being lost.
2020-10-16 16:19:11 -04:00
Alisdair McDiarmid 4f53234d8c terraform: Hide maybe-sensitive provisioner output
If the provisioner configuration includes sensitive values, it's a
reasonable assumption that we should suppress its log output. Obvious
examples where this makes sense include echoing a secret to a file using
local-exec or remote-exec.

This commit adds tests for both logging output from provisioners with
non-sensitive configuration, and suppressing logs for provisioners with
sensitive values in configuration.

Note that we do not suppress logs if connection info contains sensitive
information, as provisioners should not be logging connection
information under any circumstances.
2020-10-16 15:29:44 -04:00