Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.
The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.
The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.
Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
This is a rather-messy, complex change to get the "command" package
building again against the new backend API that was updated for
the new configuration loader.
A lot of this is mechanical rewriting to the new API, but
meta_config.go and meta_backend.go in particular saw some major
changes to interface with the new loader APIs and to deal with
the change in order of steps in the backend API.
Rather than try to modify all the hundreds of calls to the temp helper
functions, and cleanup the temp files at every call site, have all tests
work within a single temp directory that is removed at the end of
TestMain.
To avoid breaking automation where plugin-path was assumed to be set
permanently, only remove the plugin-path record if it was explicitly set
to and empty string.
Make sure the init inputFalse test actually errors from missing input,
since skipping input will still fail later during provider
initialization. We need to make sure there are two different states that
aren't a noop for migration, and reset the command struct for each run.
Also verify that we don't go into an infinite loop if there is no input.
The init command needs to parse the state to resolve providers, but
changes to the state format can cause that to fail with difficult to
understand errors. Check the terraform version during init and provide
the same error that would be returned by plan or apply.
Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.
This restores the earlier behavior of the first positional argument to
terraform init in 0.9, but as a command line option.
The positional argument was removed to improve consistency with other
commands that take a working directory as their first positional argument.
It was originally intended that this functionality would return in a
later release along with some other general improvements to Terraform's
module handling, but we're introducing here an interim solution that
uses the existing module source concept, to allow for easier porting of
workflows that previously depended on the automatic copy behavior.
In a future release this feature may change again as the module
improvements design firms up, but we expect it to be broadly compatible
with this temporary state.
Previously init would crash if given these options:
-backend=false -get-plugins=true
This is because the state is used as a source of provider dependency
information, and we need to instantiate the backend to get the state.
To avoid the crash, we now use the following adjusted behavior:
- if -backend=true, we behave as before
- if -backend=false, we instead try to instantiate the backend the same
way any other command would, without modifying its configuration
- if we're able to instantiate the backend, we use it to fetch state
for dependency resolution purposes
- if the backend is not instantiable then we assume it's not yet
configured and proceed with a nil state, which may cause us to see an
incomplete picture of the dependencies but still allows the install
to succeed. Subsequently running "terraform plan" will not work until
the backend is (re-)initialized, so the incomplete picture of required
plugins is safe.
Previously we only did this when _upgrading_, but that's unnecessarily
specific and confusing since e.g. plugins can get upgraded implicitly by
constraint changes, which would not then trigger the purge process.
Instead, we'll assume that the user is able to easily re-download plugins
that were purged here, or if they need more specific guarantees they will
manage manually a plugin directory and disable the auto-install behavior
using `-plugin-dir`.
The -plugin-dir option lets the user specify custom search paths for
plugins. This overrides all other plugin search paths, and prevents the
auto-installation of plugins.
We also make sure that the availability of plugins is always checked
during init, even if -get-plugins=false or -plugin-dir is set.
init should always write intternal data to the current directory, even
when a path is provided. The inherited behavior no longer applies to the
new use of init.
Now when -upgrade is provided to "terraform init" (and plugin installation
isn't disabled) it will:
- ignore the contents of the auto-install plugin directory when deciding
what is "available", thus causing anything there to be reinstalled,
possibly at a newer version.
- if installation completes successfully, purge from the auto-install
plugin directory any plugin-looking files that aren't in the set of
chosen plugins.
As before, plugins outside of the auto-install directory are able to
take precedence over the auto-install ones, and these will never be
upgraded nor purged.
The thinking here is that the auto-install directory is an implementation
detail directly managed by Terraform, and so it's Terraform's
responsibility to automatically keep it clean as plugins are upgraded.
We don't yet have the -plugin-dir option implemented, but once it is it
should circumvent all of this behavior and just expect providers to be
already available in the given directory, meaning that nothing will be
auto-installed, -upgraded or -purged.
Previously we had a "getProvider" function type used to implement plugin
fetching. Here we replace that with an interface type, initially with
just a "Get" function.
For now this just simplifies the interface by allowing the target
directory and protocol version to be members of the struct rather than
passed as arguments.
A later change will extend this interface to also include a method to
purge unused plugins, so that upgrading frequently doesn't leave behind
a trail of unused executable files.
As of this commit this just upgrades modules, but this option will also
later upgrade plugins and indeed anything else that's being downloaded and
installed as part of the init.
When init was modified in 0.9 to initialize a terraform working
directory, the legacy behavior was kept to copy or fetch module sources.
This left the init command without the ability that the plan and apply
commands have to target a specific directory for the operation.
This commit removes the legacy behavior altogether, and allows init to
target a directory for initialization, bringing it into parity with plan
and apply. If one want to copy a module to the target or current
directory, that will have to be done manually before calling init. We
can later reintroduce fetching modules with init without breaking this
new behavior, by adding the source as an optional second argument.
The unit tests testing the copying of sources with init have been
removed, as well as some out of date (and commented out) init tests
regarding remote states.
ConstrainVersions was documented as returning nil, but it was instead
returning an empty set. Use the Count() method to check for nil or
empty. Add test to verify failed constraints will show up as missing.
Once we've installed the necessary plugins, we'll do one more walk of
the available plugins and record the SHA256 hashes of all of the plugins
we select in the provider lock file.
The file we write here gets read when we're building ContextOpts to
initialize the main terraform context, so any command that works with
the context will then fail if any of the provider binaries change.
Previously we did plugin discovery in the main package, but as we move
towards versioned plugins we need more information available in order to
resolve plugins, so we move this responsibility into the command package
itself.
For the moment this is just preserving the existing behavior as long as
there are only internal and unversioned plugins present. This is the
final state for provisioners in 0.10, since we don't want to support
versioned provisioners yet. For providers this is just a checkpoint along
the way, since further work is required to apply version constraints from
configuration and support additional plugin search directories.
The automatic plugin discovery behavior is not desirable for tests because
we want to mock the plugins there, so we add a new backdoor for the tests
to use to skip the plugin discovery and just provide their own mock
implementations. Most of this diff is thus noisy rework of the tests to
use this new mechanism.
It's possible to not change the backend config, but require updating the
stored backend state by moving init options from the config file to the
`-backend-config` flag. If the config is the same, but the hash doesn't
match, update the stored state.
This method mirrors that of config.Backend, so we can compare the
configration of a backend read from a config vs that of a backend read
from a state. This will prevent init from reinitializing when using
`-backend-config` options that match the existing state.
The `-force-copy` option will suppress confirmation for copying state
data.
Modify some tests to use the option, making sure to leave coverage of
the Input code path.
Fixes#12749
If we merge in an extra partial config we need to recompute the hash to
compare with the old value to detect that change.
This hash needs to NOT be stored and just used as a temporary. We want
to keep the original hash in the state so that we don't detect a change
from the config (since the config will always be partial).
We need to initialize the backend even if the config has no backend set.
This allows `init` to work when unsetting a previously set backend.
Without this, there was no way to unset a backend.
Martin Atkins