Prior to our refactoring here, we were relying on a lucky coincidence for
correct behavior of the plan walk following a refresh in the same run:
- The refresh phase created placeholder objects in the state to represent
any resource instance pending creation, to allow the interpolator to
read attributes from them when evaluating "provider" and "data" blocks.
In effect, the refresh walk is creating a partial plan that only covers
creation actions, but was immediately discarding the actual diff entries
and storing only the planned new state.
- It happened that objects pending creation showed up in state with an
empty ID value, since that only gets assigned by the provider during
apply.
- The Refresh function concluded by calling terraform.State.Prune, which
deletes from the state any objects that have an empty ID value, which
therefore prevented these temporary objects from surviving into the
plan phase.
After refactoring, we no longer have this special ID field on instance
object state, and we instead rely on the Status field for tracking such
things. We also no longer have an explicit "prune" step on state, since
the state mutation methods themselves keep the structure pruned.
To address this, here we introduce a new instance object status "planned",
which is equivalent to having an empty ID value in the old world. We also
introduce a new method on states.SyncState that deletes from the state
any planned objects, which therefore replaces that portion of the old
State.prune operation just for this refresh use-case.
Finally, we are now expecting the expression evaluator to pull pending
objects from the planned changeset rather than from the state directly,
and so for correct results these placeholder resource creation changes
must also be reported in a throwaway changeset during the refresh walk.
The addition of states.ObjectPlanned also permits a previously-missing
safety check in the expression evaluator to prevent us from relying on the
incomplete value stored in state for a pending object, in the event that
some bug prevents the real pending object from being written into the
planned changeset.
Our state representation is not able to preserve unknown values, so it's
not suitable for retaining the transient incomplete values we produce
during planning.
Instead, we'll discard the unknown values when writing to state and have
the expression evaluator prefer an object from the plan where possible.
We still use the shape of the transient state to inform things like the
resource's "each mode", so the plan only masks the object values
themselves.
We're going to allow the provider to encode whatever it wants in here, so
a provider can use whatever is most convenient for its implementation
language and to avoid some of the bugs we saw with the prior model where
the forced round-trip through JSON and back into interface{} would cause
some loss of fidelity, leading to bugs.
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.
The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.
The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.
Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
Our existing core tests make extensive use of the string representation
of a state for comparison purposes, because they were written before we
began making use of helper packages like "cmp".
To avoid the need to rewrite all of those tests and potentially break
them, we will instead port that particular rendering as closely as
possible but mark it with a comment sternly warning not to use it for
anything new.
We don't want to use this moving forward for a number of reasons, but
most notably:
- printing out whole before and after state representations makes it
hard to find a subtle difference in outcome when a test fails, while
"cmp" can provide us with a real diff.
- this string serialization is constrained by the capabilities of
Terraform prior to our new state models, and so it does not
comprehensively represent all possibilities in the new world.
- it will probably behave oddly/poorly when given states containing
features that arrived after it was written, even though I made a
best effort here to make it do something reasonable in situations
I thought about.
The types here were originally written to allow us to defer decoding of
object values until schemas are available, but it turns out that this was
forcing us to defer decoding longer than necessary and potentially decode
the same value multiple times.
To avoid this, we create pairs of types to represent the encoded and
decoded versions and methods for moving between them. These types are
identical to one another apart from how the dynamic values are
represented.
In practice these pairs of functions are often used together when working
with a "full" statemgr, so these helper wrappers allow us to do that more
conveniently.
This also introduces a new interface statemgr.Storage, which represents
a state manager that has all of the storage capabilities but does not
necessarily support locking. In practice callers will usually just use
statemgr.Full, but these more-specific interfaces allow us to reflect
in APIs which subset of the statemgr functionality each function depends
on.
In the old state package we had this as a separate manager
state.BackupState, but that doesn't work with our new interfaces because
we handle lineage and serial within the state managers themselves and
don't expose them to callers anymore.
In practice it being built in to the filesystem manager is not a problem
because we only use the backup functionality for local state anyway.
This also slightly adjusts the behavior to be more intuitive. The old
BackupState relied on the implementation detail that Terraform re-persists
the original state early in an apply operation, which meant that by
coincidence it would back up the right snapshot. In this new approach,
we instead take an in-memory copy during State and then write _that_ to
disk in WriteState if the new state seems different, so we're guaranteed
that we'll always write out what we read before any changes were made.
In future we may improve this further, such as keeping multiple
generations of backups, etc. But for now this is intended to preserve the
goals of the original implementation while making its behavior
self-contained and not dependent on coincidences.
Since schemas are required to interpret provider, resource, and
provisioner attributes in configs, states, and plans, these helpers intend
to make it easier to gather up the the necessary provider types in order
to preload all of the needed schemas before beginning further processing.
Config.ProviderTypes returns directly the list of provider types, since
at this level further detail is not useful: we've not yet run the
provider allocation algorithm, and so the only thing we can reliably
extract here is provider types themselves.
State.ProviderAddrs and Plan.ProviderAddrs each return a list of
absolute provider addresses, which can then be turned into a list of
provider types using the new helper providers.AddressedTypesAbs.
Since we're already using configs.Config throughout core, this also
updates the terraform.LoadSchemas helper to use Config.ProviderTypes
to find the necessary providers, rather than implementing its own
discovery logic. states.State is not yet plumbed in, so we cannot yet
use State.ProviderAddrs to deal with the state but there's a TODO comment
to remind us to update that in a later commit when we swap out
terraform.State for states.State.
A later commit will probably refactor this further so that we can easily
obtain schema for the providers needed to interpret a plan too, but that
is deferred here because further work is required to make core work with
the new plan types first. At that point, terraform.LoadSchemas may become
providers.LoadSchemas with a different interface that just accepts lists
of provider and provisioner names that have been gathered by the caller
using these new helpers.
This is a wrapper around State that is able to perform higher-level
manipulations (at the granularity of the entire state) in a
concurrency-safe manner, using the lower-level APIs exposed by State and
all of the types it contains.
The granularity of a SyncState operation roughly matches the granularity
off a state-related EvalNode in the "terraform" package, performing a
sequence of more primitive operations while guaranteeing atomicity of the
entire change.
As a compromise for convenience of usage, it's still possible to access
the individual state data objects via this API, but they are always copied
before returning to ensure that two distinct callers cannot have data
races. Callers should access the most granular object possible for their
operation.
This idea of a "state manager" was previously modelled via the
confusingly-named state.State interface, which we've been calling a "state
manager" only in some local variable names in situations where there were
also *terraform.State variables.
As part of reworking our state models to make room for the new type
system, we also need to change what was previously the state.StateReader
interface. Since we've found the previous organization confusing anyway,
here we just copy all of those interfaces over into statemgr where we can
make the relationship to states.State hopefully a little clearer.
This is not yet a complete move of the functionality from "state", since
we're not yet ready to break existing callers. In a future commit we'll
turn the interfaces in the old "state" package into aliases of the
interfaces in this package, and update all the implementers of what will
by then be statemgr.Reader to use *states.State instead of
*terraform.State.
This also includes an adaptation of what was previously state.LocalState
into statemgr.FileSystem, using the new state serialization functionality
from package statefile instead of the old terraform.ReadState and
terraform.WriteState.
Whereas the parent directory "states" contains the models that represent
state in memory, this package's responsibility is in serializing a subset
of that data to a JSON-based file format and then reloading that data
back into memory later.
For reading, this package supports state file formats going back to
version 1, using lightly-adapted versions of the migration code previously
used in the "terraform" package. State data is upgraded to the latest
version step by step and then transformed into the in-memory state
representation, which is distinct from any of the file format structs in
this package to enable these to evolve separately.
For writing, only the latest version (4) is supported, which is a new
format that is a slightly-flattened version of the new in-memory state
models introduced in the prior commit. This format retains the outputs
from only the root module and it flattens out the module and instance
parts of the hierarchy by including the identifiers for these inside
the child object. The loader then reconstructs the multi-layer structure
we use for more convenient access in memory.
For now, the only testing in this package is of round-tripping different
versions of state through a read and a write, ensuring the output is
as desired. This exercises all of the reading, upgrading, and writing
functions but should be augmented in later commits to improve coverage
and introduce more focused tests for specific parts of the functionality.
Our previous state models in the "terraform" package had a few limitations
that are addressed here:
- Instance attributes were stored as map[string]string with dot-separated
keys representing traversals through a data structure. Now that we have
a full type system, it's preferable to store it as a real data
structure.
- The existing state structures skipped over the "resource" concept and
went straight to resource instance, requiring heuristics to decide
whether a particular resource should appear as a single object or as
a list of objects when used in configuration expressions.
- Related to the previous point, the state models also used incorrect
terminology where "ResourceState" was really a resource instance state
and "InstanceState" was really the state of a particular remote object
associated with an instance. These new models use the correct names for
each of these, introducing the idea of a "ResourceInstanceObject" as
the local record of a remote object associated with an instance.
This is a first pass at fleshing out a new model for state. Undoubtedly
there will be further iterations of this as we work on integrating these
new models into the "terraform" package.
These new model types no longer serve double-duty as a description of the
JSON state file format, since they are for in-memory use only. A
subsequent commit will introduce a separate package that deals with
persisting state to files and reloading those files later.
Martin Atkins