Commit Graph

21739 Commits

Author SHA1 Message Date
Martin Atkins 6d6a80fd9d
Update CHANGELOG.md 2017-11-01 16:58:31 -07:00
Martin Atkins 9d77d35f53
Update CHANGELOG.md 2017-11-01 16:56:59 -07:00
Martin Atkins 3da5fefdc1 command: Allow TF_DATA_DIR env var to override data directory
This allows the user to customize the location where Terraform stores
the files normally placed in the ".terraform" subdirectory, if e.g. the
current working directory is not writable.
2017-11-01 16:55:23 -07:00
Martin Atkins 4787428cad
Merge #14887: Custom diff logic for providers 2017-11-01 15:56:54 -07:00
Chris Marchesi 0c0ae3ca7c helper/schema: CustomizeDiff allowed on writable resources only
This keeps CustomizeDiff from being defined on data sources, where it
would be useless. We just catch this in InternalValidate like the rest
of the CRUD functions that are not used in data sources.
2017-11-01 15:42:24 -07:00
Chris Marchesi 2c541e8c97 helper/schema: Add behaviour detail to various custom diff comments
Added some more detailed comments to CustomizeDiff's comments. The new
comments detail how CustomizeDiff will be called in the event of
different scenarios like creating a new resource, diffing an existing
one, diffing an existing resource that has a change that requires a new
resource, and destroy/tainted resources.

Also added similar detail to ForceNew in ResourceDiff.

This should help mitigate any confusion that may come up when using
CustomizeDiff, especially in the ForceNew scenario when the second run
happens with no state.
2017-11-01 15:42:24 -07:00
James Bardin 229c70cabe
Merge pull request #16296 from stack72/manta-backend
Migrate Manta Remote state to be a backend
2017-11-01 18:07:26 -04:00
Radek Simko b599dc6471
Make CoC and support channels more visible
https://help.github.com/articles/adding-a-code-of-conduct-to-your-project/
https://help.github.com/articles/adding-support-resources-to-your-project/
2017-11-01 23:06:23 +01:00
Chris Marchesi 5031767b93 terraform: Add more exception cases for value comparison
Needed to add more cases to support value comparison exceptions that the
rest of TF expects to work (this fixes tests in various places).

Also moved things to a switch block so that it's a little more compact.
2017-11-01 14:25:32 -07:00
Chris Marchesi 12378b4ee2 terraform: Add value check to diff.Same
A diff new needs to pass basic value checks to be considered the
"same". Several provisions have been added to ensure that the list, set,
and RequiresNew behaviours that have needed some exceptions in the past
are preserved in this new logic.

This ensures that we are checking for value equality as much as
possible, which will be more important when we transition to the
possibility of diffs being sourced from external data.
2017-11-01 14:25:32 -07:00
Chris Marchesi f5bdbc0911 helper/schema: Simplify setDiff, remove exported SetDiff mentions
setDiff does not make use of its new parameter anymore, so it has been
removed. Also, as there is no more SetDiff (exported) function, mentions
of that have been removed from comments too.
2017-11-01 14:25:32 -07:00
Chris Marchesi 3ac0cdf0c0 helper/schema: Better ResourceDiff schema key validation
This fixes nil pointer issues that could come up if an invalid key was
referenced (ie: not one in the schema). Also ships a helper validation
function to streamline things.
2017-11-01 14:25:32 -07:00
Chris Marchesi 5d5a670d69 provider/test: Added complex-ish list testing
Added a list SetNew test to try and reproduce issues testing diff
customization with the Nomad provider. We are running into "diffs didn't
match during apply", with the plan diff exhibiting a strange
off-by-one-type error in a list diff:

  datacenters.#:         "1" => "2"
  datacenters.0:         "dc1" => "dc2"
  datacenters.1:         "" => "dc3"
  datacenters.2:         "" => "dc3"

The test here does not reproduce that issue, unfortunately, but should
help pinpoint the root cause through elimination.
2017-11-01 14:25:32 -07:00
Chris Marchesi 529d7e6dae helper/schema: Review -> CustomizeDiff
Restoring the naming of this field in the resource back to
CustomizeDiff, as this is generally more descriptive of the process
that's happening, despite the lengthy name.
2017-11-01 14:25:32 -07:00
Chris Marchesi 09e2109ff8 helper/schema: Resouce.Diff no longer ResourceProvider API compatible
The old comments said that this interface was API compatible with
terraform.ResourceProvider's Diff method - with the addition of passing
down meta to it, this is no longer the case.

Not too sure if this is really a big deal - schema.Resource never fully
implemented terraform.ResourceProvider, from what I can see, and the
path from Provdier.Diff to Resource.Diff is still pretty clear. Just
wanted to remove an outdated comment.
2017-11-01 14:25:32 -07:00
Chris Marchesi 9625830980 helper/schema: Move computedKeys init to init function
This simplifies the new value initialization and future-proofs it
against more complex initialization functionality.
2017-11-01 14:25:32 -07:00
Chris Marchesi f0aafe4d67 helper/schema: Restore new value for complex set test 2017-11-01 14:25:32 -07:00
Chris Marchesi 3444549d60 helper/schema: Move computed key reader logic to childAddrOf
This will make this check safer and will remove the risk of it passing
on keys with a similar prefix.
2017-11-01 14:25:32 -07:00
Chris Marchesi 36aa63b338 helper/schema: Guard against out of range on childAddrOf
This could panic if we sent a parent that was actually longer than the
child (should almost never come up, but the guard will make it safe
anyway).

Also fixed a slice style lint warning in UpdatedKeys.
2017-11-01 14:25:32 -07:00
Chris Marchesi 931b0e1441 helper/schema: Remove exported SetDiff method
The consensus is that it's a generally better idea to move setting the
functionality of old values completely to the refresh/read process,
hence it's moot to have this function around anymore. This also means we
don't need the old value reader/writer anymore, which simplifies things.
2017-11-01 14:25:32 -07:00
Chris Marchesi ad98471559 helper/schema: Correct some comments
Removed some outdated comments for SetNew, and normalized the computed
key note for SetNew, SetNewComputed, and SetDiff.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8a7c9a6f02 helper/schema: frist -> first 2017-11-01 14:25:32 -07:00
Chris Marchesi 7d5f9ed6b1 helper/schema: NewComputed values should be nil
When working on this initially, I think I thought that since NewComputed
values in the diff were empty strings, that it was using the zero value.
After review, it doesn't seem like this is the case - so I have adjusted
NewComputed to pass nil values. There is also a guard now that keeps the
new value writer from accepting computed fields with non-nil values.
2017-11-01 14:25:32 -07:00
Chris Marchesi 6f422d8c44 helper/schema: Remove unused log line
Meant to remove this before finalizing the PR. :P
2017-11-01 14:25:32 -07:00
Chris Marchesi c6647a3bb7 helper/schema: CustomizeDiff -> Review
To keep with the current convention of most other schema.Resource
functional fields being fairly short, CustomizeDiff has been changed to
"Review". It would be "Diff", however it is already used by existing
functions in schema.Provider and schema.Resource.
2017-11-01 14:25:32 -07:00
Chris Marchesi f7e42728b6 helper/schema: Remove unused Destroy check for CustomizeDiff
Both Destroy and DestroyDeposed are not propagated down the diff stack,
meaning that there is no way we can tell at this point if an instance is
being destroyed or deposed, so this check would never be used.

In this regard, Destroy never runs a diff down the stack at all, and a
deposition check is not run until *after* the provider's diff function
is called. To answer this question and close it off, we could either
determine if a resource is deposed earlier, and propagate that down, or
treat deposed resources like full destroy nodes, and not diff them at
all (but rather making a diff with the only thing in it being
DestroyDeposed flagged).
2017-11-01 14:25:32 -07:00
Chris Marchesi fa1fc2ca8e helper/schema: CustomizeDiff invocation test
Just one more test to check to make sure that CustomizeDiff is called
on resource level.
2017-11-01 14:25:32 -07:00
Chris Marchesi ee769188d7 helper/schema: More CustomizeDiff test cases
Added a few more test cases for CustomizeDiff, caught another error in
the process. I think this is ready for review now, and possibly some
real-world testing of the waters by way of porting some resources that
would benefit from the feature.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8af9610b87 helper/schema: Hook CustomizeDiffFunc into diff logic
It's alive! CustomizeDiff logic now has been inserted into the diff
process. The test_resource_with_custom_diff resource provides some basic
testing and a reference implementation.

There should now be plenty of test coverage for this feature via the
tests added for ResourceDiff, and the basic test added to the
schemaMap.Diff test, and the test resource, but more can be added to
test any specific case that comes up otherwise.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8126ee8401 helper/schema: Fix supplied config to tests 2017-11-01 14:25:32 -07:00
Chris Marchesi 196d7e63fe helper/schema: Drop ClearAll from ResourceDiff
As the interface has been specced out for ResourceDiff, the only diff
operation that can function on a non-computed key is ForceNew, and that
is only if a diff exists for that key. As such, allowing the user to
clear keys that cannot be operated on afterwards does not really make
much sense.

Will re-add this function if it is determined it's needed after all on
review.
2017-11-01 14:25:32 -07:00
Chris Marchesi 22220fd0f7 helper/schema: Final set of ResourceDiff tests, bug fixes
Final set of coverage for ResourceDiff and bug fixes to correct issues
the test cases brought up.

Will be dropping ClearAll in next commit, as with how this interface is
intended to be used, it does not really make much sense.
2017-11-01 14:25:32 -07:00
Chris Marchesi a5fc664ea6 helper/schema: Add schemaMap.DeepCopy
This provides a deep copy of a schemaMap, which will be needed for the
diff customization process as ResourceDiff will be able to flag fields
as ForceNew - we don't want to affect the source schema.
2017-11-01 14:25:32 -07:00
Chris Marchesi 6a4f7b0dce helper/schema: Don't ignore diffs for certain NewComputed keys
In diffString, removed values are marked if the old value is non-nil and
the new value is nil, regardless of if the new value is marked as a
computed value. With the new diff override behaviour, this can lead to
issues where a nil attribute may get inserted into the diff if the new
value has been marked as computed (as the value will be marked as
missing, but computed).

This propagates into finalizeDiff in some respects because even if
NewComputed is manually set in the diff that gets passed in, it is
ignored, and the schema becomes the only source of truth. Since our new
diff behaviour is mainly designed to be supported on computed keys only,
it stands to reason that we there will be a time where we want to set a
diff as NewComputed on a key, and see that change in the diff.

These two small changes makes that happen. No regressions in tests have
been observed via this change, so it seems safe and non-invasive.
2017-11-01 14:25:32 -07:00
Chris Marchesi 64cc4084d2 helper/schema: ResourceDiff tests, bug fixes
The beginning of tests for SetNew. Noticed that removed values are not
logged for computed keys in a diff - not too sure if that is going to be
a problem (possibly not as GetChange in ResourceData should still
reference state for the old value). I came on this most notably in the
"TestSetNew/complex-ish_set_diff" test, for reference.

More tests pending for other functions for coverage of other functions
and test cases as defined in #8769.
2017-11-01 14:25:32 -07:00
Chris Marchesi f5f4e0329f helper/schema: Track updated keys in ResourceDiff
This ensures that when we hook this into the main diff logic, that we
can just re-diff the keys that are modified, ensuring that the diff is
not re-run on keys that were not touched, or on keys that were
intentionally removed.
2017-11-01 14:25:32 -07:00
Chris Marchesi aeb793f968 helper/schema: Add Clear function to ResourceDiff
This should complete the feature set of the prototype. This function
removes a specific key from the existing diff, preventing conflicts.

Further functionality might be needed to make this behave as expected,
namely ensuring that we don't re-process the whole diff after the
CustomizeDiffFunc runs.
2017-11-01 14:25:32 -07:00
Chris Marchesi 1e8cfc52e9 helper/schema: Updated ResourceDiff prototype
This new prototype removes the dependence on a underlying ResourceData,
moving several items up to ensure an approach that more matches
ResourceData. Namely, we create our own MultiLevelFieldReader that
also layers updated diff values on top. New values use a small
re-implementation of the MapFieldWriter/Reader that allows computed
values to be set.

The assumption here now is that a second diff will happen after the
first one is done, processing the new values set in the 2 new
reader/writer levels and updating the diff as necessary.
2017-11-01 14:25:32 -07:00
Chris Marchesi b99c615ee6 helper/schema: New ResourceDiff object
This adds a new object, ResourceDiff, to the schema package. This
object, in conjunction with a function defined in CustomizeDiff in the
resource schema, allows for the in-flight customization of a Terraform
diff. This helps support use cases such as when there are necessary
changes to a resource that cannot be detected in config, such as via
computed fields (most of the utility in this object works on computed
fields only). It also allows for a wholesale wipe of the diff to allow
for diff logic completely offloaded to an external API, if it is a
better use case for a specific provider.

As part of this work, many internal diff functions have been moved to
use a special resourceDiffer interface, to allow for shared
functionality between ResourceDiff and ResourceData. This may be
extended to the DiffSuppressFunc as well which would restrict use of
ResourceData in DiffSuppressFunc to generally read-only fields.

This work is not yet in its final state - CustomizeDiff is not yet
implemented in the general diff workflow, new functions may be added
(notably Clear() for a single key), and functionality may be altered.
Tests will follow as well.
2017-11-01 14:25:32 -07:00
Clint e2c69d4d6e
[WIP]: website: Add CloudScale Provider link (#16512)
* add cloudscale

* update link
2017-11-01 15:05:29 -05:00
Benjamin Farley d6abdd14e7 website: correction of chunklist interpolation function example 2017-11-01 10:48:34 -07:00
Martin Atkins 505683dcd3 config/module: produce explicit error for non-registry hosts
If registry API discovery fails for a particular host then it's better to
generate an explicit error message for that early -- so we can tell the
user exactly what happened -- rather than assuming a default path and
then failing downstream when we get a 404 from that request.
2017-11-01 09:13:08 -07:00
Martin Atkins 7fa731f293 website: Modules Getting Started guide to use Terraform Registry
This is a significant rework of the Modules getting started guide to be
in terms of the Consul module available via the Terraform Registry. This
allows us to introduce the registry as part of the tutorial, and also
gives us some auto-generated documentation to link to as context for
the tutorial.

This new module is designed pretty differently than the one we formerly
used, and in particular it doesn't expose any server addresses for the
created Consul cluster -- due to using an auto-scaling group -- and thus
we're forced to use the (arguably-less-compelling) auto-scaling group name
for our output example.
2017-11-01 07:22:20 -07:00
Martin Atkins 4fa2b8c519 website: updated docs for the module registry protocol
Now includes more complete information on usage of private registries and
updates the module registry API documentation to include the new
version-enumeration endpoint along with describing Terraform's discovery
protocol.
2017-11-01 07:22:20 -07:00
Martin Atkins 56a9b1309a website: initial draft of 0.11 upgrade guide
This will be revised further as we near the final release. This contains
notes on changes that have already been merged.
2017-11-01 07:22:20 -07:00
Martin Atkins 91a4af9c8a website: rewrite modules usage documentation for new features
The modules mechanism has changed quite a bit for version 0.11 and so
although simple usage remains broadly compatible there are some
significant changes in the behavior of more complex modules.

Since large parts of this were rewritten anyway, I also took the
opportunity to do some copy-editing to make the prose on this page more
consistent with our usual editorial voice and to wrap the long
lines.
2017-11-01 07:22:20 -07:00
Martin Atkins 3ef86deb28
Update CHANGELOG.md 2017-11-01 06:58:16 -07:00
Martin Atkins 400038eda4 command: "terraform apply" uses interactive confirmation by default
In the 0.10 release we added an opt-in mode where Terraform would prompt
interactively for confirmation during apply. We made this opt-in to give
those who wrap Terraform in automation some time to update their scripts
to explicitly opt out of this behavior where appropriate.

Here we switch the default so that a "terraform apply" with no arguments
will -- if it computes a non-empty diff -- display the diff and wait for
the user to type "yes" in similar vein to the "terraform destroy" command.

This makes the commonly-used "terraform apply" a safe workflow for
interactive use, so "terraform plan" is now mainly for use in automation
where a separate planning step is used. The apply command remains
non-interactive when given an explicit plan file.

The previous behavior -- though not recommended -- can be obtained by
explicitly setting the -auto-approve option on the apply command line,
and indeed that is how all of the tests are updated here so that they can
continue to run non-interactively.
2017-11-01 06:54:39 -07:00
James Bardin 0265b1c0fe
Merge pull request #16509 from hashicorp/jbardin/get-providers
Use pooled http client for fetching providers
2017-10-31 12:44:35 -04:00
Radek Simko 586255fdc0
Merge pull request #16506 from hashicorp/b-helper-t-escape-path
helper: Escape test name in TF_LOG_PATH_MASK
2017-10-31 16:41:43 +00:00