Commit Graph

25 Commits

Author SHA1 Message Date
Martin Atkins 2c70d884d6 dag: node visits return diagnostics rather than errors
This allows node visits to also return warnings.
2018-10-16 18:44:26 -07:00
James Bardin 8cf0a8ca9c faster DAG transitive reduction
In the case of highly-connected graphs, the TransitiveReduction process
was far too computationally intensive. Since no operations are applied
to the nodes, and the walk order is not even user visible, we don't need
to sort them n^2 times.
2017-10-03 11:24:59 -04:00
Mitchell Hashimoto 6702a22074
dag: require acyclic graph 2017-02-03 21:18:34 +01:00
Mitchell Hashimoto 65752cd51a
dag: improve docs and read access during a lock 2017-02-03 21:18:33 +01:00
Mitchell Hashimoto 72a717f2de
dag: change the type sig of Update to Graph so its external friendly 2017-02-03 21:18:33 +01:00
Mitchell Hashimoto 28fff99ea8
dag: replace dag.Walk with our walker 2017-02-03 21:18:33 +01:00
James Bardin 6d30b60144 Add DebugVisitInfo
This encodes vertex debug information into the graph log when a vertex
is visited during a walk operation. These can ordered to show how the
Graph was walked.

Add a mutex to the encoder so it can be used during a parallel walk.

Moved string literal constants used for marshaling to pre-defined constants.
Did some renaming to make the marshal* structures more consistent.
2016-11-15 10:39:08 -05:00
James Bardin de0cb17a39 Add Graph.DebugOperation
The method marks the start of a set of operations on the Graph, with
extra information optionally provided in the second paramter. This
returns a function with a single End method to mark the end of the set
in the logs.

Refactor the existing graph Begin/End Operation calls to use this single
method. Remove the *string types in the marshal structs, these are
strictly informational and don't need to differentiate empty vs unset
strings.

Add calls to DebugOperation for each step while building the graph.
2016-11-14 12:47:51 -05:00
James Bardin 82b1a2abc2 Add graph transformation recording
The external api provided here is simply
dag.Graph.SetDebugWriter(io.Writer). When a writer is provided to a
Graph, it will immediately encode itself to the stream, and subsequently
encode any additional transformations to the graph. This will allow
easier logging of graph transformations without writing complete graphs
to the logs at every step. Since the marshalGraph can also be dot
encoded, this will allow translation from the JSON logs to dot graphs.
2016-11-14 08:50:34 -05:00
James Bardin 7b774f771b implement dag.Subgrapher interface
This allows the dag package to detect subgraphs, even when impelemnted
by types from other packages
2016-11-14 08:50:34 -05:00
Mitchell Hashimoto 83b1c82e93
terraform: put quotes before vertex name to avoid false panicwrap
Fixes 
2016-10-23 14:23:36 -07:00
Paul Hinze b45f53eef4
dag: fix ReverseDepthFirstWalk when nodes remove themselves
The report in  led us into a deep rabbit hole that turned out to
expose a bug in the graph walk implementation being used by the
`NoopTransformer`. The problem ended up being when two nodes in a single
dependency chain both reported `Noop() -> true` and needed to be
removed. This was breaking the walk and preventing the second node from
ever being visited.

Fixes 
2016-07-15 13:43:28 -06:00
Paul Hinze 9cd88810f4 core: log every 5s while waiting for dependencies
Helps to flush out deadlocks in the dependency graph
2015-08-10 15:50:36 -05:00
Paul Hinze baa33d7326 core: dag errors should cascade to all descendents
We weren't marking skipped nodes as failing, so any
grandchild-and-deeper dependencies would still evaluate.

For example:

    A -> B -> C -> D

If B failed, C would be skipped, but D would still be evaluated.

This fixes the behavior so C, D, and any further descendents will all be
skipped when B fails.

Addresses crashing aspect of  and likely a lot of other confusing
failure modes.
2015-08-07 11:38:58 -05:00
Paul Hinze ce49dd6080 core: graph command gets -verbose and -draw-cycles
When you specify `-verbose` you'll get the whole graph of operations,
which gives a better idea of the operations terraform performs and in
what order.

The DOT graph is now generated with a small internal library instead of
simple string building. This allows us to ensure the graph generation is
as consistent as possible, among other benefits.

We set `newrank = true` in the graph, which I've found does just as good
a job organizing things visually as manually attempting to rank the nodes
based on depth.

This also fixes `-module-depth`, which was broken post-AST refector.
Modules are now expanded into subgraphs with labels and borders. We
have yet to regain the plan graphing functionality, so I removed that
from the docs for now.

Finally, if `-draw-cycles` is added, extra colored edges will be drawn
to indicate the path of any cycles detected in the graph.

A notable implementation change included here is that
{Reverse,}DepthFirstWalk has been made deterministic. (Before it was
dependent on `map` ordering.) This turned out to be unnecessary to gain
determinism in the final DOT-level implementation, but it seemed
a desirable enough of a property that I left it in.
2015-04-27 09:23:47 -05:00
Paul Hinze 97acccd3ed core: targeted operations
Add `-target=resource` flag to core operations, allowing users to
target specific resources in their infrastructure. When `-target` is
used, the operation will only apply to that resource and its
dependencies.

The calculated dependencies are different depending on whether we're
running a normal operation or a `terraform destroy`.

Generally, "dependencies" refers to ancestors: resources falling
_before_ the target in the graph, because their changes are required to
accurately act on the target.

For destroys, "dependencies" are descendents: those resources which fall
_after_ the target. These resources depend on our target, which is going
to be destroyed, so they should also be destroyed.
2015-03-31 14:49:38 -05:00
Mitchell Hashimoto 865de51816 dag: do a DFS for each vertex 2015-02-27 19:37:59 -08:00
Mitchell Hashimoto ed2075e384 dag: TransitiveReduction 2015-02-27 19:12:19 -08:00
Mitchell Hashimoto 54fd742ef6 dag: walk should be able to be halted 2015-02-19 12:07:57 -08:00
Mitchell Hashimoto 5b0004ffc7 dag: detect self references, use multierror 2015-02-19 12:07:56 -08:00
Mitchell Hashimoto e86698c50d dag: fix walk order issue, scc issues 2015-02-19 12:07:55 -08:00
Mitchell Hashimoto d59ced3c57 dag: Validate for DAGs 2015-02-19 12:07:55 -08:00
Mitchell Hashimoto e94c43e0dc dag: Walk 2015-02-19 12:07:55 -08:00
Mitchell Hashimoto cfa3d89265 dag: find root of AcyclicGraph 2015-02-19 12:07:55 -08:00
Mitchell Hashimoto 21e4501edb terraform: start implementing interfaces for semantic checks 2015-02-19 12:07:52 -08:00