Commit Graph

27557 Commits

Author SHA1 Message Date
James Bardin f8893785f0 separate core and provider loggers
Now that hclog can independently set levels on related loggers, we can
separate the log levels for different subsystems in terraform.

This adds the new environment variables, `TF_LOG_CORE` and
`TF_LOG_PROVIDER`, which each take the same set of log level arguments,
and only applies to logs from that subsystem. This means that setting
`TF_LOG_CORE=level` will not show logs from providers, and
`TF_LOG_PROVIDER=level` will not show logs from core. The behavior of
`TF_LOG` alone does not change.

While it is not necessarily needed since the default is to disable logs,
there is also a new level argument of `off`, which reflects the
associated level in hclog.
2020-10-23 12:46:32 -04:00
Pam Selle c9e362bb5f
Merge pull request #26653 from hashicorp/pselle/getresource
Return marked After values in GetResource
2020-10-23 12:28:03 -04:00
James Bardin 37d57a2593
Merge pull request #26687 from hashicorp/jbardin/warning-errors
do not return warnings as errors from eval
2020-10-23 12:18:27 -04:00
James Bardin 0d4cee2309
Merge pull request #26628 from hashicorp/jbardin/backport-26613
0.14 backport of 26613
2020-10-23 12:01:00 -04:00
James Bardin 820e641b97 do not return warnings as errors from eval
Warnings alone cannot be returned from eval nodes, since are still
treated as errors.
2020-10-23 10:39:30 -04:00
James Bardin ce9e4b2ebd
Merge pull request #26674 from hashicorp/jbardin/refresh-orphan
Read orphaned instances during plan
2020-10-23 08:59:15 -04:00
James Bardin b4f9c54c4d update hclog
Added an Off level to hclog, so we can individually disable logging at
various levels.

Added IndependentLevels so that sublogger levels are not linked to their
parents.
2020-10-22 15:26:24 -04:00
Petros Kolyvas bfbdbe9bba
Tool-specific provisioner docs change to red notice box (#26681) 2020-10-22 15:50:46 -03:00
craiggenner 6408533fca
The index must be non-negative integer
and added instructions on how to get the last value in the list.
2020-10-22 19:03:27 +01:00
Jakson Rodrigues 818029826b
Fixed config argument on example (#26304) 2020-10-22 14:08:25 -03:00
Kerim Satirli f4790d8f61
updates `description` to reflect endpoint usage (#26278) 2020-10-22 13:56:11 -03:00
Thanonchai f3fa59f65d
Update config.html.md (#26622)
* Update config.html.md

When reading this page, I couldn't find the list of the "supported backends to the left".  They're actually on a different page, so thought I'd update it so that others wouldn't find it confusing like me.

If this is ok with you, would it be possible to label this PR with 'hacktoberfest-accepted'?  I'm still new to this.  If not, I'd be alright.  Thank you!

* Update config.html.md

Swapped the full URL in the link for a relative path

Co-authored-by: Petros Kolyvas <petros@hashicorp.com>
2020-10-22 13:49:40 -03:00
Justin Long 0ce11faffd
Clear wording that bucket must pre-exist (#26276)
Experienced similar issue as https://github.com/hashicorp/terraform/issues/18417 this updates the documentation so that it's more clear a storage bucket must exist prior to configuring the backend.
2020-10-22 13:35:02 -03:00
Alisdair McDiarmid fe9a9fadfb internal: Suppress duplicate version constraints
A set of version constraints can contain duplicates. This can happen if
multiple identical constraints are specified throughout a configuration.

When rendering the set, it is confusing to display redundant
constraints. This commit changes the string renderer to only show the
first instance of a given constraint, and adds unit tests for this
function to cover this change.

This also fixes a bug with the locks file generation: previously, a
configuration with redundant constraints would result in this error on
second init:

Error: Invalid provider version constraints

  on .terraform.lock.hcl line 6:
  (source code not available)

The recorded version constraints for provider
registry.terraform.io/hashicorp/random must be written in normalized form:
"3.0.0".
2020-10-22 12:08:00 -04:00
James Bardin bc1a841d65
Merge pull request #26665 from hashicorp/jbardin/logging
Restore "crash.log" behavior and remove prefixedio
2020-10-22 10:34:51 -04:00
James Bardin 73627e4dc3 don't read data sources that are to be removed
We can directly remove orphaned data sources from the refesh state.
2020-10-22 10:15:22 -04:00
James Bardin 72e81de0fc update tests
Some tests could not handle reading orphaned resources. It also turns
out the ReadResource mock never returned the correct state in the
default case at all.
2020-10-22 09:46:42 -04:00
James Bardin ddb2bbf4e9 Read orphaned resources during plan
This forces orphaned resources to be re-read during planning, removing
them from the state if they no longer exist.

This needs to be done for a bare `refresh` execution, since Terraform
should remove instances that don't exist and are not in the
configuration from the state. They should also be removed from state so
there is no Delete change planned, as not all providers will gracefully
handle a delete operation on a resource that does not exist.
2020-10-22 09:46:42 -04:00
James Bardin ef4fee5acb warnings are now on stderr 2020-10-21 18:24:09 -04:00
James Bardin c2af5333e8 use a log sink to capture logs for panicwrap
Use a separate log sink to always capture trace logs for the panicwrap
handler to write out in a crash log.

This requires creating a log file in the outer process and passing that
path to the child process to log to.
2020-10-21 17:29:07 -04:00
James Bardin b61488a8ba write traceback to log crash log 2020-10-21 17:29:07 -04:00
James Bardin a2dee91eba update hclog-go
We need some fixes from master.
2020-10-21 17:29:07 -04:00
James Bardin fd4f7eb0b9 remove prefixed io
The main process is now handling what output to print, so it doesn't do
any good to try and run it through prefixedio, which is only adding
extra coordination to echo the same data.
2020-10-21 17:29:07 -04:00
Jerry Chong 2f091836c9
Modified terraform get command (#26465)
-Added PATH
-Added -no-color option
2020-10-21 18:14:54 -03:00
Diod FR f0edb192b3
ADD CLI option position for force-unlock command (#26626)
* ADD CLI option position for force-unlock command

* Update force-unlock.html.markdown

Made a change to also include the missing [DIR]

Co-authored-by: Petros Kolyvas <petros@hashicorp.com>
2020-10-21 18:13:18 -03:00
Pam Selle c6be76e53c Return marked After values in GetResource
If a change exists for a resource instance,
the After value is returned, however, this value
will not have its marks as it as been encoded.
This Marks the return value so the marks follow
that resource reference.
2020-10-21 16:18:54 -04:00
Alisdair McDiarmid bc769aeb7c
Update CHANGELOG.md 2020-10-21 14:12:48 -04:00
Bishwa Shrestha c41336bc77
Exit with error if UI input scan fails (#26509) 2020-10-21 14:10:06 -04:00
Martin Atkins 70c52c778c
lang/funcs: textencodebase64 and textdecodebase64 2020-10-21 11:09:25 -07:00
Martin Atkins 1dc4950bfa lang/funcs: Rename the base64 character encoding functions
These were initially introduced as functions with "encode" and "decode"
prefixes, but that doesn't match with our existing convention of putting
the encoding format first so that the encode and decode functions will
group together in a alphabetically-ordered function list.

"text" is not really a defined serialization format, but it's a short word
that hopefully represents well enough what these functions are aiming to
encode and decode, while being consistent with existing functions like
jsonencode/jsondecode, yamlencode/yamldecode, etc.

The "base64" at the end here is less convincing because there is precedent
for that modifier to appear both at the beginning and the end in our
existing function names. I chose to put it at the end here because that
seems to be our emergent convention for situations where the base64
encoding is a sort of secondary modifier alongside the primary purpose
of the function, as we see with "filebase64". (base64gzip is an exception
here, but it seems outvoted by the others.)
2020-10-21 10:56:56 -07:00
James Bardin 1d9d82973b move panicwrap handler to logging package 2020-10-21 13:47:16 -04:00
r0bnet 877399c631 lang/funcs: Functions for encoding text in specific character encodings 2020-10-21 10:39:43 -07:00
Alisdair McDiarmid 7a31e56cb7
Merge pull request #26637 from hashicorp/alisdair/fix-locksfile-unconstrained-versions
internal: Fix lockfile constraint output for 1.2.*
2020-10-21 12:05:19 -04:00
Martin Atkins 1fe314b70b getproviders: Don't log if EvalSymlinks changes nothing
Previously this codepath was generating a confusing message in the absense
of any symlinks, because filepath.EvalSymlinks returns a successful result
if the target isn't a symlink.

Now we'll emit the log line only if filepath.EvalSymlinks returns a
result that's different in a way that isn't purely syntactic (which
filepath.Clean would "fix").

The new message is a little more generic because technically we've not
actually ensured that a difference here was caused by a symlink and so
we shouldn't over-promise and generate something potentially misleading.
2020-10-21 07:49:11 -07:00
Kristin Laemmert f211cf0c73 build: update comments in dockerfile 2020-10-21 08:49:36 -04:00
Kristin Laemmert 9d623290f4 command/e2etest: do not leave bad directories behind
If you run the e2etests locally and use a configured plugin_cache_dir,
the test will leave a bad directory behind in your cache dir that causes
later `init`s to fail. To circumvent this, pass an explicity-empty CLI
config file.

This is a nicety for local developers and not necessarily required, but
it happens to me often enough that I'd like to fix it. It's probably not
a *bad* idea to pass an explicit cli config to all e2etests, honestly,
but this is the only one that causes active problems so I limited this
PR to that one test.

Here's the error which occurs on subsequent `init` if this test is run on a
machine that uses a plugin cache dir:

2020/10/13 10:41:05 [TRACE] providercache.fillMetaCache: error while scanning directory /Users/mildwonkey/.terraform.d/plugin-cache: failed to read metadata about /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: stat /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: no such file or directory
2020-10-21 07:53:28 -04:00
Martin Atkins 2332a7ab47 replacefile: don't create the temporary file in TMPDIR
ioutil.TempFile has a special case where an empty string for its dir
argument is interpreted as a request to automatically look up the system
temporary directory, which is commonly /tmp .

We don't want that behavior here because we're specifically trying to
create the temporary file in the same directory as the file we're hoping
to replace. If the file gets created in /tmp then it might be on a
different device and thus the later atomic rename won't work.

Instead, we'll add our own special case to explicitly use "." when the
given filename is in the current working directory. That overrides the
special automatic behavior of ioutil.TempFile and thus forces the
behavior we need.

This hadn't previously mattered for earlier callers of this code because
they were creating files in subdirectories, but this codepath was failing
for the dependency lock file due to it always being created directly
in the current working directory.

Unfortunately since this is a picky implementation detail I couldn't find
a good way to write a unit test for it without considerable refactoring.
Instead, I verified manually that the temporary filename wasn't in /tmp on
my Linux system, and hope that the comment inline will explain this
situation well enough to avoid an accidental regression in future
maintenence.
2020-10-20 16:29:59 -07:00
Paul Thrasher 960042ad42
Merge pull request #26618 from hashicorp/pault/tfce-continue-on-error
Allow runs to continue on tfce error
2020-10-20 13:23:09 -07:00
Alisdair McDiarmid 9904667fc3
Merge pull request #26644 from hashicorp/alisdair/ignore-changes-sensitive
terraform: Fix sensitive values in ignore changes
2020-10-20 14:22:09 -04:00
Pam Selle e448a12c11
Merge pull request #26647 from hashicorp/pselle/codecov
[build] Only run codecov on master
2020-10-20 14:09:45 -04:00
Pam Selle df65de6945 Only run codecov on master 2020-10-20 13:54:48 -04:00
Pam Selle 7924117acc
Add note to upgrade guide about provider sensitivity (#26643)
* Add note to upgrade guide about provider sensitivity

Now that sensitivity follows attributes providers mark
as sensitive, add this note to the upgrade guide.

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-10-20 13:13:51 -04:00
Alisdair McDiarmid fb98fc98fa terraform: Fix sensitive values in ignore changes
Because ignore_changes configuration can refer to resource arguments
which are assigned sensitive values, we need to unmark the resource
object before processing.
2020-10-20 12:27:17 -04:00
Alisdair McDiarmid 9576a5b2d8 internal: Fix lockfile constraint output for 1.2.*
If a configuration requires a partial provider version (with some parts
unspecified), Terraform considers this as a constrained-to-zero version.
For example, a version constraint of 1.2 will result in an attempt to
install version 1.2.0, even if 1.2.1 is available.

When writing the dependency locks file, we previously would write 1.2.*,
as this is the in-memory representation of 1.2. This would then cause an
error on re-reading the locks file, as this is not a valid constraint
format.

Instead, we now explicitly convert the constraint to its zero-filled
representation before writing the locks file. This ensures that it
correctly round-trips.

Because this change is made in getproviders.VersionConstraintsString, it
also affects the output of the providers sub-command.
2020-10-20 10:14:03 -04:00
James Bardin d9ac57ffae
Merge pull request #26632 from hashicorp/jbardin/logging
Structured logging
2020-10-20 09:23:57 -04:00
James Bardin 18f676ea11
Merge pull request #26638 from hashicorp/jbardin/assert-compat-dynamic-blocks
Treat empty strings as null in NestingSet objs when looking for dynamic blocks in AssertObjectCompatible
2020-10-20 09:06:04 -04:00
James Bardin ef086399f9 compare empty strings as null in sets
The Legacy SDK cannot handle missing strings from objects in sets, and
will insert an empty string when planning the missing value. This
subverts the `couldHaveUnknownBlockPlaceholder` check, and causes
errors when `dynamic` is used with NestingSet blocks.

We don't have a separate codepath to handle the internals of
AssertObjectCompatible differently for the legacy SDK, but we can treat
empty strings as null strings within set objects to avoid the failed
assertions.
2020-10-19 18:07:45 -04:00
Pam Selle cdebf2820d
Merge pull request #26590 from hashicorp/pselle/sensitivity-providers
Mark attributes providers mark as sensitive
2020-10-19 16:13:39 -04:00
Pam Selle 394e60608c Allocate new copies of paths to avoid append drama
Create new copies of the Path to avoid possible append
related dramas. Also add a test to cover nested block
within blocks
2020-10-19 15:24:14 -04:00
James Bardin a3a20e0396 remove LevelFilter
We want to always be using the hclogger to filter whenever possible
2020-10-19 14:31:05 -04:00