Commit Graph

27140 Commits

Author SHA1 Message Date
James Bardin eb2d4434c8 logging env variable docs 2020-10-23 12:46:32 -04:00
James Bardin f8893785f0 separate core and provider loggers
Now that hclog can independently set levels on related loggers, we can
separate the log levels for different subsystems in terraform.

This adds the new environment variables, `TF_LOG_CORE` and
`TF_LOG_PROVIDER`, which each take the same set of log level arguments,
and only applies to logs from that subsystem. This means that setting
`TF_LOG_CORE=level` will not show logs from providers, and
`TF_LOG_PROVIDER=level` will not show logs from core. The behavior of
`TF_LOG` alone does not change.

While it is not necessarily needed since the default is to disable logs,
there is also a new level argument of `off`, which reflects the
associated level in hclog.
2020-10-23 12:46:32 -04:00
James Bardin b4f9c54c4d update hclog
Added an Off level to hclog, so we can individually disable logging at
various levels.

Added IndependentLevels so that sublogger levels are not linked to their
parents.
2020-10-22 15:26:24 -04:00
James Bardin bc1a841d65
Merge pull request #26665 from hashicorp/jbardin/logging
Restore "crash.log" behavior and remove prefixedio
2020-10-22 10:34:51 -04:00
James Bardin ef4fee5acb warnings are now on stderr 2020-10-21 18:24:09 -04:00
James Bardin c2af5333e8 use a log sink to capture logs for panicwrap
Use a separate log sink to always capture trace logs for the panicwrap
handler to write out in a crash log.

This requires creating a log file in the outer process and passing that
path to the child process to log to.
2020-10-21 17:29:07 -04:00
James Bardin b61488a8ba write traceback to log crash log 2020-10-21 17:29:07 -04:00
James Bardin a2dee91eba update hclog-go
We need some fixes from master.
2020-10-21 17:29:07 -04:00
James Bardin fd4f7eb0b9 remove prefixed io
The main process is now handling what output to print, so it doesn't do
any good to try and run it through prefixedio, which is only adding
extra coordination to echo the same data.
2020-10-21 17:29:07 -04:00
Jerry Chong 2f091836c9
Modified terraform get command (#26465)
-Added PATH
-Added -no-color option
2020-10-21 18:14:54 -03:00
Diod FR f0edb192b3
ADD CLI option position for force-unlock command (#26626)
* ADD CLI option position for force-unlock command

* Update force-unlock.html.markdown

Made a change to also include the missing [DIR]

Co-authored-by: Petros Kolyvas <petros@hashicorp.com>
2020-10-21 18:13:18 -03:00
Alisdair McDiarmid bc769aeb7c
Update CHANGELOG.md 2020-10-21 14:12:48 -04:00
Bishwa Shrestha c41336bc77
Exit with error if UI input scan fails (#26509) 2020-10-21 14:10:06 -04:00
Martin Atkins 70c52c778c
lang/funcs: textencodebase64 and textdecodebase64 2020-10-21 11:09:25 -07:00
Martin Atkins 1dc4950bfa lang/funcs: Rename the base64 character encoding functions
These were initially introduced as functions with "encode" and "decode"
prefixes, but that doesn't match with our existing convention of putting
the encoding format first so that the encode and decode functions will
group together in a alphabetically-ordered function list.

"text" is not really a defined serialization format, but it's a short word
that hopefully represents well enough what these functions are aiming to
encode and decode, while being consistent with existing functions like
jsonencode/jsondecode, yamlencode/yamldecode, etc.

The "base64" at the end here is less convincing because there is precedent
for that modifier to appear both at the beginning and the end in our
existing function names. I chose to put it at the end here because that
seems to be our emergent convention for situations where the base64
encoding is a sort of secondary modifier alongside the primary purpose
of the function, as we see with "filebase64". (base64gzip is an exception
here, but it seems outvoted by the others.)
2020-10-21 10:56:56 -07:00
James Bardin 1d9d82973b move panicwrap handler to logging package 2020-10-21 13:47:16 -04:00
r0bnet 877399c631 lang/funcs: Functions for encoding text in specific character encodings 2020-10-21 10:39:43 -07:00
Alisdair McDiarmid 7a31e56cb7
Merge pull request #26637 from hashicorp/alisdair/fix-locksfile-unconstrained-versions
internal: Fix lockfile constraint output for 1.2.*
2020-10-21 12:05:19 -04:00
Martin Atkins 1fe314b70b getproviders: Don't log if EvalSymlinks changes nothing
Previously this codepath was generating a confusing message in the absense
of any symlinks, because filepath.EvalSymlinks returns a successful result
if the target isn't a symlink.

Now we'll emit the log line only if filepath.EvalSymlinks returns a
result that's different in a way that isn't purely syntactic (which
filepath.Clean would "fix").

The new message is a little more generic because technically we've not
actually ensured that a difference here was caused by a symlink and so
we shouldn't over-promise and generate something potentially misleading.
2020-10-21 07:49:11 -07:00
Kristin Laemmert f211cf0c73 build: update comments in dockerfile 2020-10-21 08:49:36 -04:00
Kristin Laemmert 9d623290f4 command/e2etest: do not leave bad directories behind
If you run the e2etests locally and use a configured plugin_cache_dir,
the test will leave a bad directory behind in your cache dir that causes
later `init`s to fail. To circumvent this, pass an explicity-empty CLI
config file.

This is a nicety for local developers and not necessarily required, but
it happens to me often enough that I'd like to fix it. It's probably not
a *bad* idea to pass an explicit cli config to all e2etests, honestly,
but this is the only one that causes active problems so I limited this
PR to that one test.

Here's the error which occurs on subsequent `init` if this test is run on a
machine that uses a plugin cache dir:

2020/10/13 10:41:05 [TRACE] providercache.fillMetaCache: error while scanning directory /Users/mildwonkey/.terraform.d/plugin-cache: failed to read metadata about /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: stat /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: no such file or directory
2020-10-21 07:53:28 -04:00
Martin Atkins 2332a7ab47 replacefile: don't create the temporary file in TMPDIR
ioutil.TempFile has a special case where an empty string for its dir
argument is interpreted as a request to automatically look up the system
temporary directory, which is commonly /tmp .

We don't want that behavior here because we're specifically trying to
create the temporary file in the same directory as the file we're hoping
to replace. If the file gets created in /tmp then it might be on a
different device and thus the later atomic rename won't work.

Instead, we'll add our own special case to explicitly use "." when the
given filename is in the current working directory. That overrides the
special automatic behavior of ioutil.TempFile and thus forces the
behavior we need.

This hadn't previously mattered for earlier callers of this code because
they were creating files in subdirectories, but this codepath was failing
for the dependency lock file due to it always being created directly
in the current working directory.

Unfortunately since this is a picky implementation detail I couldn't find
a good way to write a unit test for it without considerable refactoring.
Instead, I verified manually that the temporary filename wasn't in /tmp on
my Linux system, and hope that the comment inline will explain this
situation well enough to avoid an accidental regression in future
maintenence.
2020-10-20 16:29:59 -07:00
Paul Thrasher 960042ad42
Merge pull request #26618 from hashicorp/pault/tfce-continue-on-error
Allow runs to continue on tfce error
2020-10-20 13:23:09 -07:00
Alisdair McDiarmid 9904667fc3
Merge pull request #26644 from hashicorp/alisdair/ignore-changes-sensitive
terraform: Fix sensitive values in ignore changes
2020-10-20 14:22:09 -04:00
Pam Selle e448a12c11
Merge pull request #26647 from hashicorp/pselle/codecov
[build] Only run codecov on master
2020-10-20 14:09:45 -04:00
Pam Selle df65de6945 Only run codecov on master 2020-10-20 13:54:48 -04:00
Pam Selle 7924117acc
Add note to upgrade guide about provider sensitivity (#26643)
* Add note to upgrade guide about provider sensitivity

Now that sensitivity follows attributes providers mark
as sensitive, add this note to the upgrade guide.

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-10-20 13:13:51 -04:00
Alisdair McDiarmid fb98fc98fa terraform: Fix sensitive values in ignore changes
Because ignore_changes configuration can refer to resource arguments
which are assigned sensitive values, we need to unmark the resource
object before processing.
2020-10-20 12:27:17 -04:00
Alisdair McDiarmid 9576a5b2d8 internal: Fix lockfile constraint output for 1.2.*
If a configuration requires a partial provider version (with some parts
unspecified), Terraform considers this as a constrained-to-zero version.
For example, a version constraint of 1.2 will result in an attempt to
install version 1.2.0, even if 1.2.1 is available.

When writing the dependency locks file, we previously would write 1.2.*,
as this is the in-memory representation of 1.2. This would then cause an
error on re-reading the locks file, as this is not a valid constraint
format.

Instead, we now explicitly convert the constraint to its zero-filled
representation before writing the locks file. This ensures that it
correctly round-trips.

Because this change is made in getproviders.VersionConstraintsString, it
also affects the output of the providers sub-command.
2020-10-20 10:14:03 -04:00
James Bardin d9ac57ffae
Merge pull request #26632 from hashicorp/jbardin/logging
Structured logging
2020-10-20 09:23:57 -04:00
James Bardin 18f676ea11
Merge pull request #26638 from hashicorp/jbardin/assert-compat-dynamic-blocks
Treat empty strings as null in NestingSet objs when looking for dynamic blocks in AssertObjectCompatible
2020-10-20 09:06:04 -04:00
James Bardin ef086399f9 compare empty strings as null in sets
The Legacy SDK cannot handle missing strings from objects in sets, and
will insert an empty string when planning the missing value. This
subverts the `couldHaveUnknownBlockPlaceholder` check, and causes
errors when `dynamic` is used with NestingSet blocks.

We don't have a separate codepath to handle the internals of
AssertObjectCompatible differently for the legacy SDK, but we can treat
empty strings as null strings within set objects to avoid the failed
assertions.
2020-10-19 18:07:45 -04:00
Pam Selle cdebf2820d
Merge pull request #26590 from hashicorp/pselle/sensitivity-providers
Mark attributes providers mark as sensitive
2020-10-19 16:13:39 -04:00
Pam Selle 394e60608c Allocate new copies of paths to avoid append drama
Create new copies of the Path to avoid possible append
related dramas. Also add a test to cover nested block
within blocks
2020-10-19 15:24:14 -04:00
James Bardin a3a20e0396 remove LevelFilter
We want to always be using the hclogger to filter whenever possible
2020-10-19 14:31:05 -04:00
James Bardin 0b31ffa587 use a single log writer
Use a single log writer instance for all std library logging.

Setup the std log writer in the logging package, and remove boilerplate
from test packages.
2020-10-19 14:29:54 -04:00
James Bardin abf6b9b378 get properly configured hcloggers
make sure plugins get hcloggers configured to match core
2020-10-19 14:29:54 -04:00
James Bardin 8a7838266e remove old plugin config
This is unused and leads to confusion
2020-10-19 14:29:54 -04:00
James Bardin 211edf5d75 use hclog as the default logger
Inject hclog as the default logger in the main binary.
2020-10-19 14:29:54 -04:00
James Bardin 776482cc59 update hclog 2020-10-19 14:29:52 -04:00
James Bardin 6ca477f042 move helper/logging to internal
remove a dead code file too
2020-10-19 14:27:53 -04:00
Alisdair McDiarmid 5e047b0a0b
Merge pull request #26611 from hashicorp/alisdair/sensitive-values-provisioners
Fixes for sensitive values used as input to provisioners
2020-10-19 13:39:18 -04:00
Alisdair McDiarmid 9d2994ab9b website: Sensitive values in provisioner config 2020-10-19 13:19:51 -04:00
Pam Selle b44dad3ce6
Merge pull request #26623 from hashicorp/pselle/cleanup
Comment fixing
2020-10-19 10:30:40 -04:00
Alisdair McDiarmid 24edf66c01
Update CHANGELOG.md 2020-10-19 10:06:07 -04:00
Alisdair McDiarmid e6d42ec4b3
Update CHANGELOG.md 2020-10-19 10:05:59 -04:00
Alisdair McDiarmid 4206b69060
Merge pull request #26588 from hashicorp/alisdair/upgrade-colorable
go get github.com/mattn/go-colorable@v0.1.8
2020-10-19 10:04:07 -04:00
James Bardin 488840171a
Merge pull request #26613 from hashicorp/jbardin/plugin-dynamic-json
Accept JSON encoded dynamic values from providers
2020-10-19 09:37:50 -04:00
James Bardin 353937411d handle empty json 2020-10-19 09:31:37 -04:00
Pam Selle 02c48f8071 Comment fixing 2020-10-18 13:00:09 -04:00