Every provisioner that uses communicator implements its own retryFunc.
Take the remote-exec implementation (since it's the most complete) and
put it in the communicator package for each provisioner to use.
Add a public interface `communicator.Fatal`, which can wrap an error to
indicate a fatal error that should not be retried.
It's becoming more common for users to have many ssh keys loaded into an
agent, and with the default max auth attempts of an openssh server at 6,
one often needs to specify which id to use in order to avoid a `too many
authentication failures` error.
Add a connection field called `agent_identity` which will function
similarly to the ssh_config IdentityFile when used in conjunction with
an ssh agent. This uses `agent_identity` rather than `identity_file` to
specify that the file is not used directly for authentication, rather
it's used to choose which identity returned from the agent to
authenticate with first.
This feature tries a number of different methods to match the agent
identity. First the provisioner attempts to read the id file and extract
the public key. If that isn't available, we look for a .pub authorized
key file. Either of these will result in a public key that can be
matched directly against the agent keys. Finally we fall back to
matching the comment string exactly, and the id as a suffix. The only
result of using the agent_identity is the reordering of the public keys
used for authentication, and if there is no exact match the client
will still attempt remaining keys until there is an error.
It appears that the cacert option for the winrm provisioner was
not getting passed correctly to the winrm package. Log output
showed that CACert was false regardless of configuration.
While the validation of the connector looked for cacert, the winrm
communicator looked for ca_cert.
This changeset performs the following:
- Updates `masterzen/winrm` vendor to include change from (https://github.com/masterzen/winrm/pull/73)
- Updates `dylanmei/winrmtest` vendor to include change from (https://github.com/dylanmei/winrmtest/pull/4)
- Updates `packer-community/winrmcp` vendor to include the removal of the `masterzen/winrm/winrm` sub-class as a result of the `winrm` CLI tool being removed from the `masterzen/winrm` repository.
- Changes `communicator/winrm/communicator.go` to conform to the new ABI in the `masterzen/winrm` library.
This should completely remove any LGPL licensed dependencies inside of the Terraform project.
```
$ make test
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/08/20 13:40:16 Generated command/internal_plugin_list.go
go test -i $(go list ./... | grep -v '/terraform/vendor/' | grep -v '/builtin/bins/') || exit 1
echo $(go list ./... | grep -v '/terraform/vendor/' | grep -v '/builtin/bins/') | \
xargs -t -n4 go test -timeout=60s -parallel=4
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform github.com/hashicorp/terraform/backend github.com/hashicorp/terraform/backend/atlas github.com/hashicorp/terraform/backend/init
ok github.com/hashicorp/terraform 0.011s
ok github.com/hashicorp/terraform/backend 0.020s
ok github.com/hashicorp/terraform/backend/atlas 0.634s
ok github.com/hashicorp/terraform/backend/init 0.007s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/backend/legacy github.com/hashicorp/terraform/backend/local github.com/hashicorp/terraform/backend/remote-state github.com/hashicorp/terraf
orm/backend/remote-state/azure
ok github.com/hashicorp/terraform/backend/legacy 0.009s
ok github.com/hashicorp/terraform/backend/local 0.211s
ok github.com/hashicorp/terraform/backend/remote-state 0.006s
ok github.com/hashicorp/terraform/backend/remote-state/azure 0.010s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/backend/remote-state/consul github.com/hashicorp/terraform/backend/remote-state/inmem github.com/hashicorp/terraform/backend/remote-state/s
3 github.com/hashicorp/terraform/backend/remote-state/swift
ok github.com/hashicorp/terraform/backend/remote-state/consul 0.007s
ok github.com/hashicorp/terraform/backend/remote-state/inmem 0.013s
ok github.com/hashicorp/terraform/backend/remote-state/s3 0.007s
ok github.com/hashicorp/terraform/backend/remote-state/swift 0.013s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/builtin/providers/test github.com/hashicorp/terraform/builtin/provisioners/chef github.com/hashicorp/terraform/builtin/provisioners/file gi
thub.com/hashicorp/terraform/builtin/provisioners/local-exec
ok github.com/hashicorp/terraform/builtin/providers/test 1.544s
ok github.com/hashicorp/terraform/builtin/provisioners/chef 0.017s
ok github.com/hashicorp/terraform/builtin/provisioners/file 0.006s
ok github.com/hashicorp/terraform/builtin/provisioners/local-exec 0.078s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/builtin/provisioners/remote-exec github.com/hashicorp/terraform/builtin/provisioners/salt-masterless github.com/hashicorp/terraform/command
github.com/hashicorp/terraform/command/clistate
ok github.com/hashicorp/terraform/builtin/provisioners/remote-exec 1.037s
ok github.com/hashicorp/terraform/builtin/provisioners/salt-masterless 0.008s
ok github.com/hashicorp/terraform/command 14.589s
? github.com/hashicorp/terraform/command/clistate [no test files]
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/command/e2etest github.com/hashicorp/terraform/command/format github.com/hashicorp/terraform/communicator github.com/hashicorp/terraform/co
mmunicator/remote
ok github.com/hashicorp/terraform/command/e2etest 3.729s
ok github.com/hashicorp/terraform/command/format 0.004s
ok github.com/hashicorp/terraform/communicator 0.005s
ok github.com/hashicorp/terraform/communicator/remote 0.003s [no tests to run]
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/communicator/shared github.com/hashicorp/terraform/communicator/ssh github.com/hashicorp/terraform/communicator/winrm github.com/hashicorp/
terraform/config
ok github.com/hashicorp/terraform/communicator/shared 0.007s
ok github.com/hashicorp/terraform/communicator/ssh 0.016s
ok github.com/hashicorp/terraform/communicator/winrm 0.018s
ok github.com/hashicorp/terraform/config 0.213s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/config/module github.com/hashicorp/terraform/dag github.com/hashicorp/terraform/digraph github.com/hashicorp/terraform/flatmap
ok github.com/hashicorp/terraform/config/module 0.044s
ok github.com/hashicorp/terraform/dag 0.010s
ok github.com/hashicorp/terraform/digraph 0.002s
ok github.com/hashicorp/terraform/flatmap 0.002s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/acctest github.com/hashicorp/terraform/helper/config github.com/hashicorp/terraform/helper/copy github.com/hashicorp/terraform/helpe
r/diff
? github.com/hashicorp/terraform/helper/acctest [no test files]
ok github.com/hashicorp/terraform/helper/config 0.005s
? github.com/hashicorp/terraform/helper/copy [no test files]
ok github.com/hashicorp/terraform/helper/diff 0.005s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/encryption github.com/hashicorp/terraform/helper/experiment github.com/hashicorp/terraform/helper/hashcode github.com/hashicorp/terr
aform/helper/hilmapstructure
? github.com/hashicorp/terraform/helper/encryption [no test files]
ok github.com/hashicorp/terraform/helper/experiment 0.001s
ok github.com/hashicorp/terraform/helper/hashcode 0.001s
? github.com/hashicorp/terraform/helper/hilmapstructure [no test files]
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/logging github.com/hashicorp/terraform/helper/mutexkv github.com/hashicorp/terraform/helper/pathorcontents github.com/hashicorp/terr
aform/helper/resource
? github.com/hashicorp/terraform/helper/logging [no test files]
ok github.com/hashicorp/terraform/helper/mutexkv 0.055s
ok github.com/hashicorp/terraform/helper/pathorcontents 0.002s
ok github.com/hashicorp/terraform/helper/resource 2.659s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/schema github.com/hashicorp/terraform/helper/shadow github.com/hashicorp/terraform/helper/signalwrapper github.com/hashicorp/terrafo
rm/helper/slowmessage
ok github.com/hashicorp/terraform/helper/schema 0.063s
ok github.com/hashicorp/terraform/helper/shadow 0.156s
ok github.com/hashicorp/terraform/helper/signalwrapper 0.022s
ok github.com/hashicorp/terraform/helper/slowmessage 0.102s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/structure github.com/hashicorp/terraform/helper/validation github.com/hashicorp/terraform/helper/variables github.com/hashicorp/terr
aform/helper/wrappedreadline
ok github.com/hashicorp/terraform/helper/structure 0.004s
ok github.com/hashicorp/terraform/helper/validation 0.004s
ok github.com/hashicorp/terraform/helper/variables 0.006s
? github.com/hashicorp/terraform/helper/wrappedreadline [no test files]
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/wrappedstreams github.com/hashicorp/terraform/moduledeps github.com/hashicorp/terraform/plugin github.com/hashicorp/terraform/plugin
/discovery
? github.com/hashicorp/terraform/helper/wrappedstreams [no test files]
ok github.com/hashicorp/terraform/moduledeps 0.004s
ok github.com/hashicorp/terraform/plugin 0.046s
ok github.com/hashicorp/terraform/plugin/discovery 0.029s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/repl github.com/hashicorp/terraform/scripts github.com/hashicorp/terraform/state github.com/hashicorp/terraform/state/remote
ok github.com/hashicorp/terraform/repl 0.006s
ok github.com/hashicorp/terraform/scripts 0.008s
ok github.com/hashicorp/terraform/state 2.617s
ok github.com/hashicorp/terraform/state/remote 0.025s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/terraform github.com/hashicorp/terraform/tools/terraform-bundle
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/terraform github.com/hashicorp/terraform/tools/terraform-bundle
ok github.com/hashicorp/terraform/terraform 4.222s
? github.com/hashicorp/terraform/tools/terraform-bundle [no test files]
```
These are skipped without the SSH_UPLOAD_TEST flag, since they require
connecting to a running ssh server on localhost as the current user, and
one creates a very large file.
Fixes#10463
I'm really surprised this flew under the radar for years...
By having unique PRNGs, the SSH communicator could and would
generate identical ScriptPaths and two provisioners running in parallel
could overwrite each other and execute the same script. This would
happen because they're both seeded by the current time which could
potentially be identical if done in parallel...
Instead, we share the rand now so that the sequence is guaranteed
unique. As an extra measure of robustness, we also multiple by the PID
so that we're also protected against two processes at the same time.
Fixes#3605 and adds the functionality suggested in PR #7440.
This PR is using a different appraoch that (IMHO) feels cleaner and (even more important) adds support for Windows at the same time.
Without a seed, the "random" script path locations for the remote-exec
provisioner were actually deterministic!
Every rand.Int31() would return the same pseudorandom chain starting w/
the numbers: 1298498081, 2019727887, 1427131847, 939984059, ...
So here we properly seed the communicators so the script paths are
actually random, and multiple runs on a single remote host have much
less chance of clobbering each other.
Fixes#4186
Kudos to @DustinChaloupka for the correct hunch leading to this fix!
As the command completes, the winrm.Command does not wait for its copy of
the remote.Command streams to report itself completed.
This adds an additional sync.WaitGroup to ensure that copy is finished up
before moving on, solving the data race.
Fixes the following data race:
```
==================
WARNING: DATA RACE
Read by goroutine 6:
github.com/hashicorp/terraform/communicator/winrm.TestStart()
/Users/phinze/go/src/github.com/hashicorp/terraform/communicator/winrm/communicator_test.go:79 +0xa44
testing.tRunner()
/private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:456 +0xdc
Previous write by goroutine 14:
bytes.(*Buffer).ReadFrom()
/private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/bytes/buffer.go:174 +0x465
io.copyBuffer()
/private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/io/io.go:375 +0x1a5
io.Copy()
/private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/io/io.go:351 +0x78
Goroutine 6 (running) created at:
testing.RunTests()
/private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:561 +0xaa3
testing.(*M).Run()
/private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:494 +0xe4
main.main()
github.com/hashicorp/terraform/communicator/winrm/_test/_testmain.go:62 +0x20f
Goroutine 14 (finished) created at:
github.com/hashicorp/terraform/communicator/winrm.runCommand()
/Users/phinze/go/src/github.com/hashicorp/terraform/communicator/winrm/communicator.go:151 +0xf8
==================
```
We've been moving away from config fields expecting file paths that
Terraform will load, instead prefering fields that expect file contents,
leaning on `file()` to do loading from a path.
This helps with consistency and also flexibility - since this makes it
easier to shift sensitive files into environment variables.
Here we add a little helper package to manage the transitional period
for these fields where we support both behaviors.
Also included is the first of several fields being shifted over - SSH
private keys in provisioner connection config.
We're moving to new field names so the behavior is more intuitive, so
instead of `key_file` it's `private_key` now.
Additional field shifts will be included in follow up PRs so they can be
reviewed and discussed individually.
On connections where no second hop is made, there's no problem if the
agent forwarding connection is denied, so we shouldn't treat that as a
fatal error.
In the SSH client configuration, we had SSH Agent authentication listed
before the static PrivateKey loaded from the `key_file` setting.
Switching the default of the `agent` setting exposed the fact that the
SSH agent overrides the `key_file` during the handshake. By listing the
`key_file` first, we catch the provided key before any query goes out to
the agent.
Adds a key-based authentication SSH test to cover this new behavior. It
fails without the reordering on any machine with an SSH agent running.
Fixes#2614
* adds `bastion_*` fields to `connection` which add configuration for a
bastion host
* if `bastion_host` is set, connect to that host first, then jump
through it to make the SSH connection to `host`
* enables SSH Agent forwarding by default
This changes SSH Agent utilization from opt-in to opt-out, bringing
Terraform in line with the behavior of Packer and the `ssh` command
itself.
But skip SSH agent connections if the SSH_AUTH_SOCK env var doesn't exist,
whcih means there's no agent to connect to.
I added a debug log line in the last commit, only to find out it’s now
logging the same info twice. So removed the double entry and tweaked
the existing once.
While fixing the vet warning also tried to improve any feedback by
showing the actual output/error instead of just the error code.
While testing this I noticed only adding stderr output is not enough as
not all error info is send to stderr, but sometimes also to stdout.
Pretty nice test coverage this way, covering all WinRM actions/logic by
using the winrmtest package. I had to extend/update/fix that package a
little here and there, but it now serves a nice purpose for testing
WinRM stuff…
* We now return an error when you set the script_path to
C:\Windows\Temp explaining this is currently not supported
* The fix in PR #1588 is converted to the updated setup in this PR
including the unit tests
Last thing to do is add a few tests for the WinRM communicator…
This is needed as preperation for adding WinRM support. There is still
one error in the tests which needs another look, but other than that it
seems like were now ready to start working on the WinRM part…
James Bardin