ResiLien
/
terraform
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
terraform/.github/workflows/build.yml

481 lines
17 KiB
YAML
Raw Normal View History

Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
name: Build Terraform CLI Packages
# If you want to test changes to this file before merging to a main branch,
# push them up to a branch whose name has the prefix "build-workflow-dev/",
# which is a special prefix that triggers this workflow even though it's not
# actually a release branch.
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
# NOTE: This workflow is currently used only to verify that all commits to a
# release branch are buildable. It's set up to generate some artifacts that
# might in principle be consumed by a downstream release process, but currently
# they are not used in this way and official Terraform CLI releases are instead
# built using a separate process maintained elsewhere. We intend to adopt this
# new process fully later, once other HashiCorp-internal tooling is ready.
#
# Currently this process produces what should be working packages but packages
# NOT suitable for distribution to end-users as official releases, because it
# doesn't include a step to ensure that "terraform version" (and similar) will
# report the intended version number. Consequently we can safely use these
# results for testing purposes, but not yet for release purposes. See the
# "build" job below for a FIXME comment related to version numbers.
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
on:
workflow_dispatch:
push:
branches:
- main
- 'v[0-9]+.[0-9]+'
- build-workflow-dev/*
tags:
- 'v[0-9]+.[0-9]+.[0-9]+*'
env:
PKG_NAME: "terraform"
build: Constrain permissions for the "build" workflow steps This workflow only generates artifacts and doesn't need to modify anything about the repository.
2022-01-03 19:45:08 +01:00
permissions:
contents: read
statuses: write
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
jobs:
get-product-version:
name: "Determine intended Terraform version"
runs-on: ubuntu-latest
outputs:
product-version: ${{ steps.get-product-version.outputs.product-version }}
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Need all commits and tags to find a reasonable version number
build: Build and run e2etest as part of the release build pipeline This uses the decoupled build and run strategy to run the e2etests so that we can arrange to run the tests against the real release packages produced elsewhere in this workflow, rather than ones generated just in time by the test harness. The modifications to make-archive.sh here make it more consistent with its originally-intended purpose of producing a harness for testing "real" release executables. Our earlier compromise of making it include its own terraform executable came from a desire to use that script as part of manual cross-platform testing when we weren't yet set up to support automation of those tests as we're doing here. That does mean, however, that the terraform-e2etest package content must be combined with content from a terraform release package in order to produce a valid contest for running the tests. We use a single job to cross-compile the test harness for all of the supported platforms, because that build is relatively fast and so not worth the overhead of matrix build, but then use a matrix build to actually run the tests so that we can run them in a worker matching the target platform. We currently have access only to amd64 (x64) runners in GitHub Actions and so for the moment this process is limited only to the subset of our supported platforms which use that architecture.
2021-12-23 18:10:59 +01:00
- name: Git Describe
id: git-describe
run: |
git describe --first-parent
echo "::set-output name=raw-version::$(git describe --first-parent)"
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
- name: Decide version number
id: get-product-version
build: Build and run e2etest as part of the release build pipeline This uses the decoupled build and run strategy to run the e2etests so that we can arrange to run the tests against the real release packages produced elsewhere in this workflow, rather than ones generated just in time by the test harness. The modifications to make-archive.sh here make it more consistent with its originally-intended purpose of producing a harness for testing "real" release executables. Our earlier compromise of making it include its own terraform executable came from a desire to use that script as part of manual cross-platform testing when we weren't yet set up to support automation of those tests as we're doing here. That does mean, however, that the terraform-e2etest package content must be combined with content from a terraform release package in order to produce a valid contest for running the tests. We use a single job to cross-compile the test harness for all of the supported platforms, because that build is relatively fast and so not worth the overhead of matrix build, but then use a matrix build to actually run the tests so that we can run them in a worker matching the target platform. We currently have access only to amd64 (x64) runners in GitHub Actions and so for the moment this process is limited only to the subset of our supported platforms which use that architecture.
2021-12-23 18:10:59 +01:00
shell: bash
env:
RAW_VERSION: ${{ steps.git-describe.outputs.raw-version }}
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
run: |
build: Build and run e2etest as part of the release build pipeline This uses the decoupled build and run strategy to run the e2etests so that we can arrange to run the tests against the real release packages produced elsewhere in this workflow, rather than ones generated just in time by the test harness. The modifications to make-archive.sh here make it more consistent with its originally-intended purpose of producing a harness for testing "real" release executables. Our earlier compromise of making it include its own terraform executable came from a desire to use that script as part of manual cross-platform testing when we weren't yet set up to support automation of those tests as we're doing here. That does mean, however, that the terraform-e2etest package content must be combined with content from a terraform release package in order to produce a valid contest for running the tests. We use a single job to cross-compile the test harness for all of the supported platforms, because that build is relatively fast and so not worth the overhead of matrix build, but then use a matrix build to actually run the tests so that we can run them in a worker matching the target platform. We currently have access only to amd64 (x64) runners in GitHub Actions and so for the moment this process is limited only to the subset of our supported platforms which use that architecture.
2021-12-23 18:10:59 +01:00
echo "::set-output name=product-version::${RAW_VERSION#v}"
- name: Report chosen version number
run: |
[ -n "${{steps.get-product-version.outputs.product-version}}" ]
echo "::notice title=Terraform CLI Version::${{ steps.get-product-version.outputs.product-version }}"
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
get-go-version:
name: "Determine Go toolchain version"
runs-on: ubuntu-latest
outputs:
go-version: ${{ steps.get-go-version.outputs.go-version }}
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
steps:
- uses: actions/checkout@v2
- name: Determine Go version
id: get-go-version
# We use .go-version as our source of truth for current Go
# version, because "goenv" can react to it automatically.
run: |
echo "Building with Go $(cat .go-version)"
echo "::set-output name=go-version::$(cat .go-version)"
generate-metadata-file:
name: "Generate release metadata"
runs-on: ubuntu-latest
build: Consolidate build-darwin job into just "build" We can use an extra matrix dimension to select which execution environment we'll use for each GOOS/GOARCH pair, and thus avoid duplicating the job definition for darwin just to set runs-on: macos-latest for it. This is not really an intended use of a matrix dimension because it's directly related to the existing "goos" one, rather than being an independent third dimension, but it doesn't matter in practice because we're using the "include" option to specify exact combinations, and thus we're not relying on the built-in functionality to generate all possible matrix combinations.
2022-01-03 19:27:42 +01:00
needs: get-product-version
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
outputs:
filepath: ${{ steps.generate-metadata-file.outputs.filepath }}
build: Consolidate build-darwin job into just "build" We can use an extra matrix dimension to select which execution environment we'll use for each GOOS/GOARCH pair, and thus avoid duplicating the job definition for darwin just to set runs-on: macos-latest for it. This is not really an intended use of a matrix dimension because it's directly related to the existing "goos" one, rather than being an independent third dimension, but it doesn't matter in practice because we're using the "include" option to specify exact combinations, and thus we're not relying on the built-in functionality to generate all possible matrix combinations.
2022-01-03 19:27:42 +01:00
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
steps:
- uses: actions/checkout@v2
- name: Generate package metadata
id: generate-metadata-file
uses: hashicorp/actions-generate-metadata@main
with:
version: ${{ needs.get-product-version.outputs.product-version }}
product: ${{ env.PKG_NAME }}
- uses: actions/upload-artifact@v2
with:
name: metadata.json
path: ${{ steps.generate-metadata-file.outputs.filepath }}
build:
build: Consolidate build-darwin job into just "build" We can use an extra matrix dimension to select which execution environment we'll use for each GOOS/GOARCH pair, and thus avoid duplicating the job definition for darwin just to set runs-on: macos-latest for it. This is not really an intended use of a matrix dimension because it's directly related to the existing "goos" one, rather than being an independent third dimension, but it doesn't matter in practice because we're using the "include" option to specify exact combinations, and thus we're not relying on the built-in functionality to generate all possible matrix combinations.
2022-01-03 19:27:42 +01:00
name: Build for ${{ matrix.goos }}_${{ matrix.goarch }}
runs-on: ${{ matrix.runson }}
needs:
- get-product-version
- get-go-version
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
strategy:
matrix:
include:
build: Consolidate build-darwin job into just "build" We can use an extra matrix dimension to select which execution environment we'll use for each GOOS/GOARCH pair, and thus avoid duplicating the job definition for darwin just to set runs-on: macos-latest for it. This is not really an intended use of a matrix dimension because it's directly related to the existing "goos" one, rather than being an independent third dimension, but it doesn't matter in practice because we're using the "include" option to specify exact combinations, and thus we're not relying on the built-in functionality to generate all possible matrix combinations.
2022-01-03 19:27:42 +01:00
- {goos: "freebsd", goarch: "386", runson: "ubuntu-latest"}
- {goos: "freebsd", goarch: "amd64", runson: "ubuntu-latest"}
- {goos: "freebsd", goarch: "arm", runson: "ubuntu-latest"}
- {goos: "linux", goarch: "386", runson: "ubuntu-latest"}
- {goos: "linux", goarch: "amd64", runson: "ubuntu-latest"}
- {goos: "linux", goarch: "arm", runson: "ubuntu-latest"}
- {goos: "linux", goarch: "arm64", runson: "ubuntu-latest"}
- {goos: "openbsd", goarch: "386", runson: "ubuntu-latest"}
- {goos: "openbsd", goarch: "amd64", runson: "ubuntu-latest"}
- {goos: "solaris", goarch: "amd64", runson: "ubuntu-latest"}
- {goos: "windows", goarch: "386", runson: "ubuntu-latest"}
- {goos: "windows", goarch: "amd64", runson: "ubuntu-latest"}
- {goos: "darwin", goarch: "amd64", runson: "macos-latest"}
- {goos: "darwin", goarch: "arm64", runson: "macos-latest"}
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
fail-fast: false
steps:
- uses: actions/checkout@v2
- name: Install Go toolchain
uses: actions/setup-go@v2
with:
go-version: ${{ needs.get-go-version.outputs.go-version }}
build: Consolidate build-darwin job into just "build" We can use an extra matrix dimension to select which execution environment we'll use for each GOOS/GOARCH pair, and thus avoid duplicating the job definition for darwin just to set runs-on: macos-latest for it. This is not really an intended use of a matrix dimension because it's directly related to the existing "goos" one, rather than being an independent third dimension, but it doesn't matter in practice because we're using the "include" option to specify exact combinations, and thus we're not relying on the built-in functionality to generate all possible matrix combinations.
2022-01-03 19:27:42 +01:00
# FIXME: We're not currently setting the hard-coded version string in
# version/version.go at any point here, which means that the packages
# this process builds are not suitable for release. Once we're using
# Go 1.18 we may begin using the version information automatically
# embedded by the Go toolchain, at which point we won't need any
# special steps during build, but failing that we'll need to rework
# the version/version.go package so we can more readily update it
# using linker flags rather than direct code modification.
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
- name: Build
env:
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
build: CGO_ENABLED when building for macOS Normally when we cross-compile we do so without CGo, because we don't have suitable C headers available for systems other than the host. However, building for macOS on macOS is special because there are sufficient headers available on darwin_amd64 to build for both darwin_amd64 _and_ darwin_arm64. Also, we _must_ use CGo on macOS because the system resolver is only available via darwin's libc, and so building without CGo produces executables that don't resolve hostnames correctly. This is a conditional in bash to avoid having to duplicate the entire step. Perhaps later we'll find a more general version of this which can avoid the special case, but this is sufficient for the moment.
2022-01-06 00:10:05 +01:00
ACTIONSOS: ${{ matrix.runson }}
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
run: |
mkdir dist out
build: CGO_ENABLED when building for macOS Normally when we cross-compile we do so without CGo, because we don't have suitable C headers available for systems other than the host. However, building for macOS on macOS is special because there are sufficient headers available on darwin_amd64 to build for both darwin_amd64 _and_ darwin_arm64. Also, we _must_ use CGo on macOS because the system resolver is only available via darwin's libc, and so building without CGo produces executables that don't resolve hostnames correctly. This is a conditional in bash to avoid having to duplicate the entire step. Perhaps later we'll find a more general version of this which can avoid the special case, but this is sufficient for the moment.
2022-01-06 00:10:05 +01:00
if [ "$ACTIONSOS" == "macos-latest" ] && [ "$GOOS" == "darwin" ]; then
# When building for macOS _on_ macOS we must force CGo to get
# correct hostname resolution behavior. (This must be conditional
# because other cross-compiles won't have suitable headers
# available to use CGo; darwin_amd64 has suitable headers to
# cross-build for darwin_arm64.)
export CGO_ENABLED=1
fi
build: Consolidate build-darwin job into just "build" We can use an extra matrix dimension to select which execution environment we'll use for each GOOS/GOARCH pair, and thus avoid duplicating the job definition for darwin just to set runs-on: macos-latest for it. This is not really an intended use of a matrix dimension because it's directly related to the existing "goos" one, rather than being an independent third dimension, but it doesn't matter in practice because we're using the "include" option to specify exact combinations, and thus we're not relying on the built-in functionality to generate all possible matrix combinations.
2022-01-03 19:27:42 +01:00
go build -ldflags "-w -s" -o dist/ .
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
zip -r -j out/${{ env.PKG_NAME }}_${{ needs.get-product-version.outputs.product-version }}_${{ matrix.goos }}_${{ matrix.goarch }}.zip dist/
- uses: actions/upload-artifact@v2
with:
name: ${{ env.PKG_NAME }}_${{ needs.get-product-version.outputs.product-version }}_${{ matrix.goos }}_${{ matrix.goarch }}.zip
path: out/${{ env.PKG_NAME }}_${{ needs.get-product-version.outputs.product-version }}_${{ matrix.goos }}_${{ matrix.goarch }}.zip
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
package-linux:
name: "Build Linux distro packages for ${{ matrix.arch }}"
runs-on: ubuntu-latest
needs:
- get-product-version
- build
strategy:
matrix:
include:
- {arch: "386"}
- {arch: "amd64"}
- {arch: "arm"}
- {arch: "arm64"}
fail-fast: false
env:
os: linux
arch: ${{matrix.arch}}
version: ${{needs.get-product-version.outputs.product-version}}
steps:
- name: "Download Terraform CLI package"
uses: actions/download-artifact@v2
id: clipkg
with:
name: terraform_${{ env.version }}_${{ env.os }}_${{ env.arch }}.zip
path: .
- name: Extract packages
run: |
mkdir -p dist
(cd dist && unzip "../terraform_${{ env.version }}_${{ env.os }}_${{ env.arch }}.zip")
mkdir -p out
- name: Build Linux distribution packages
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
uses: hashicorp/package@v1
with:
name: "terraform"
description: "Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned."
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
arch: ${{ matrix.arch }}
version: ${{ env.version }}
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
maintainer: "HashiCorp"
homepage: "https://terraform.io/"
license: "MPL-2.0"
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
binary: "dist/terraform"
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
deb_depends: "git"
rpm_depends: "git"
- name: Gather Linux distribution package filenames
run: |
echo "RPM_PACKAGE=$(basename out/*.rpm)" >> $GITHUB_ENV
echo "DEB_PACKAGE=$(basename out/*.deb)" >> $GITHUB_ENV
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
- name: "Save .rpm package"
uses: actions/upload-artifact@v2
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
with:
name: ${{ env.RPM_PACKAGE }}
path: out/${{ env.RPM_PACKAGE }}
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
- name: "Save .deb package"
uses: actions/upload-artifact@v2
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
with:
name: ${{ env.DEB_PACKAGE }}
path: out/${{ env.DEB_PACKAGE }}
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
# TODO: homebrew packages for macOS
#package-homebrew:
# name: Build Homebrew package for darwin_${{ matrix.arch }}
# runs-on: macos-latest
# needs:
# - get-product-version
# - build
# strategy:
# matrix:
# arch: ["amd64", "arm64"]
# fail-fast: false
# ...
package-docker:
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
name: Build Docker image for linux_${{ matrix.arch }}
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
runs-on: ubuntu-latest
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
needs:
- get-product-version
- build
strategy:
matrix:
arch: ["amd64"]
fail-fast: false
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
Initial work on a new package build process For the moment this is just an experimental additional sidecar package build process, separate from the one we really use for releases, so that we can get some experience building in the GitHub Actions environment before hopefully eventually switching to using the artifacts from this process as the packages we'll release through the official release channels. It will react to any push to one of our release branches or to a release tag by building official-release-like .zip, .deb, and .rpm packages, along with Docker images, based on the content of the corresponding commit. For the moment this doesn't actually produce _shippable_ packages because in particular it doesn't know how to update our version/version.go file to hard-code the correct version number. Once Go 1.18 is release and we've upgraded to it we'll switch to using debug.ReadBuildInfo to determine our version number at runtime and so no longer need to directly update a source file for each release, but that functionality isn't yet available in our current Go 1.17 release.
2021-12-23 01:47:15 +01:00
env:
repo: ${{github.event.repository.name}}
version: ${{needs.get-product-version.outputs.product-version}}
steps:
- uses: actions/checkout@v2
- name: Build Docker images
uses: hashicorp/actions-docker-build@v1
with:
version: ${{env.version}}
target: default
arch: ${{matrix.arch}}
dockerfile: .github/workflows/build-Dockerfile
tags: |
docker.io/hashicorp/${{env.repo}}:${{env.version}}
986891699432.dkr.ecr.us-east-1.amazonaws.com/hashicorp/${{env.repo}}:${{env.version}}
build: Build and run e2etest as part of the release build pipeline This uses the decoupled build and run strategy to run the e2etests so that we can arrange to run the tests against the real release packages produced elsewhere in this workflow, rather than ones generated just in time by the test harness. The modifications to make-archive.sh here make it more consistent with its originally-intended purpose of producing a harness for testing "real" release executables. Our earlier compromise of making it include its own terraform executable came from a desire to use that script as part of manual cross-platform testing when we weren't yet set up to support automation of those tests as we're doing here. That does mean, however, that the terraform-e2etest package content must be combined with content from a terraform release package in order to produce a valid contest for running the tests. We use a single job to cross-compile the test harness for all of the supported platforms, because that build is relatively fast and so not worth the overhead of matrix build, but then use a matrix build to actually run the tests so that we can run them in a worker matching the target platform. We currently have access only to amd64 (x64) runners in GitHub Actions and so for the moment this process is limited only to the subset of our supported platforms which use that architecture.
2021-12-23 18:10:59 +01:00
e2etest-build:
name: Build e2etest for ${{ matrix.goos }}_${{ matrix.goarch }}
runs-on: ubuntu-latest
build: Separate Linux distro package builds into separate job In our build workflow we'll treat Linux distribution packaging (currently .deb and .rpm packages) as a separate job, instead of embedding it into the "build" job, so that this step can happen concurrently with the other derived actions like the docker image build, and the e2etest runs.
2022-01-03 20:20:24 +01:00
needs: ["get-go-version"]
build: Build and run e2etest as part of the release build pipeline This uses the decoupled build and run strategy to run the e2etests so that we can arrange to run the tests against the real release packages produced elsewhere in this workflow, rather than ones generated just in time by the test harness. The modifications to make-archive.sh here make it more consistent with its originally-intended purpose of producing a harness for testing "real" release executables. Our earlier compromise of making it include its own terraform executable came from a desire to use that script as part of manual cross-platform testing when we weren't yet set up to support automation of those tests as we're doing here. That does mean, however, that the terraform-e2etest package content must be combined with content from a terraform release package in order to produce a valid contest for running the tests. We use a single job to cross-compile the test harness for all of the supported platforms, because that build is relatively fast and so not worth the overhead of matrix build, but then use a matrix build to actually run the tests so that we can run them in a worker matching the target platform. We currently have access only to amd64 (x64) runners in GitHub Actions and so for the moment this process is limited only to the subset of our supported platforms which use that architecture.
2021-12-23 18:10:59 +01:00
strategy:
matrix:
# We build test harnesses only for the v1.0 Compatibility Promises
# supported platforms. Even within that set, we can only run on
# architectures for which we have GitHub Actions runners available,
# which is currently only amd64 (x64).
# TODO: GitHub Actions does support _self-hosted_ arm and arm64
# runners, so we could potentially run some ourselves to run our
# tests there, but at the time of writing there is no documented
# support for darwin_arm64 (macOS on Apple Silicon).
include:
- {goos: "darwin", goarch: "amd64"}
#- {goos: "darwin", goarch: "arm64"}
- {goos: "windows", goarch: "amd64"}
- {goos: "linux", goarch: "amd64"}
#- {goos: "linux", goarch: "arm"}
#- {goos: "linux", goarch: "arm64"}
fail-fast: false
env:
build_script: ./internal/command/e2etest/make-archive.sh
steps:
- uses: actions/checkout@v2
- name: Install Go toolchain
uses: actions/setup-go@v2
with:
go-version: ${{ needs.get-go-version.outputs.go-version }}
- name: Build test harness package
env:
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
run: |
bash ./internal/command/e2etest/make-archive.sh
- uses: actions/upload-artifact@v2
with:
name: terraform-e2etest_${{ matrix.goos }}_${{ matrix.goarch }}.zip
path: internal/command/e2etest/build/terraform-e2etest_${{ matrix.goos }}_${{ matrix.goarch }}.zip
if-no-files-found: error
e2etest-linux:
name: e2etest for linux_${{ matrix.goarch }}
runs-on: ubuntu-latest
needs:
- get-product-version
- build
- e2etest-build
strategy:
matrix:
include:
- {goarch: "amd64"}
#- {goarch: "arm64"}
#- {goarch: "arm"}
fail-fast: false
env:
os: linux
arch: ${{ matrix.goarch }}
version: ${{needs.get-product-version.outputs.product-version}}
steps:
# NOTE: This intentionally _does not_ check out the source code
# for the commit/tag we're building, because by now we should
# have everything we need in the combination of CLI release package
# and e2etest package for this platform. (This helps ensure that we're
# really testing the release package and not inadvertently testing a
# fresh build from source.)
- name: "Download e2etest package"
uses: actions/download-artifact@v2
id: e2etestpkg
with:
name: terraform-e2etest_${{ env.os }}_${{ env.arch }}.zip
path: .
- name: "Download Terraform CLI package"
uses: actions/download-artifact@v2
id: clipkg
with:
name: terraform_${{env.version}}_${{ env.os }}_${{ env.arch }}.zip
path: .
- name: Extract packages
run: |
unzip "./terraform-e2etest_${{ env.os }}_${{ env.arch }}.zip"
unzip "./terraform_${{env.version}}_${{ env.os }}_${{ env.arch }}.zip"
- name: Run E2E Tests
run: |
TF_ACC=1 ./e2etest -test.v
e2etest-darwin:
name: e2etest for darwin_${{ matrix.goarch }}
runs-on: macos-latest
needs:
- get-product-version
build: Consolidate build-darwin job into just "build" We can use an extra matrix dimension to select which execution environment we'll use for each GOOS/GOARCH pair, and thus avoid duplicating the job definition for darwin just to set runs-on: macos-latest for it. This is not really an intended use of a matrix dimension because it's directly related to the existing "goos" one, rather than being an independent third dimension, but it doesn't matter in practice because we're using the "include" option to specify exact combinations, and thus we're not relying on the built-in functionality to generate all possible matrix combinations.
2022-01-03 19:27:42 +01:00
- build
build: Build and run e2etest as part of the release build pipeline This uses the decoupled build and run strategy to run the e2etests so that we can arrange to run the tests against the real release packages produced elsewhere in this workflow, rather than ones generated just in time by the test harness. The modifications to make-archive.sh here make it more consistent with its originally-intended purpose of producing a harness for testing "real" release executables. Our earlier compromise of making it include its own terraform executable came from a desire to use that script as part of manual cross-platform testing when we weren't yet set up to support automation of those tests as we're doing here. That does mean, however, that the terraform-e2etest package content must be combined with content from a terraform release package in order to produce a valid contest for running the tests. We use a single job to cross-compile the test harness for all of the supported platforms, because that build is relatively fast and so not worth the overhead of matrix build, but then use a matrix build to actually run the tests so that we can run them in a worker matching the target platform. We currently have access only to amd64 (x64) runners in GitHub Actions and so for the moment this process is limited only to the subset of our supported platforms which use that architecture.
2021-12-23 18:10:59 +01:00
- e2etest-build
strategy:
matrix:
include:
- {goarch: "amd64"}
#- {goarch: "arm64"}
fail-fast: false
env:
os: darwin
arch: ${{ matrix.goarch }}
version: ${{needs.get-product-version.outputs.product-version}}
steps:
# NOTE: This intentionally _does not_ check out the source code
# for the commit/tag we're building, because by now we should
# have everything we need in the combination of CLI release package
# and e2etest package for this platform. (This helps ensure that we're
# really testing the release package and not inadvertently testing a
# fresh build from source.)
- name: "Download e2etest package"
uses: actions/download-artifact@v2
id: e2etestpkg
with:
name: terraform-e2etest_${{ env.os }}_${{ env.arch }}.zip
path: .
- name: "Download Terraform CLI package"
uses: actions/download-artifact@v2
id: clipkg
with:
name: terraform_${{env.version}}_${{ env.os }}_${{ env.arch }}.zip
path: .
- name: Extract packages
run: |
unzip "./terraform-e2etest_${{ env.os }}_${{ env.arch }}.zip"
unzip "./terraform_${{env.version}}_${{ env.os }}_${{ env.arch }}.zip"
- name: Run E2E Tests
run: |
TF_ACC=1 ./e2etest -test.v
e2etest-windows:
name: e2etest for windows_${{ matrix.goarch }}
runs-on: windows-latest
needs:
- get-product-version
- build
- e2etest-build
strategy:
matrix:
include:
- {goarch: "amd64"}
fail-fast: false
env:
os: windows
arch: ${{ matrix.goarch }}
version: ${{needs.get-product-version.outputs.product-version}}
steps:
# NOTE: This intentionally _does not_ check out the source code
# for the commit/tag we're building, because by now we should
# have everything we need in the combination of CLI release package
# and e2etest package for this platform. (This helps ensure that we're
# really testing the release package and not inadvertently testing a
# fresh build from source.)
- name: "Download e2etest package"
uses: actions/download-artifact@v2
id: e2etestpkg
with:
name: terraform-e2etest_${{ env.os }}_${{ env.arch }}.zip
path: .
- name: "Download Terraform CLI package"
uses: actions/download-artifact@v2
id: clipkg
with:
name: terraform_${{env.version}}_${{ env.os }}_${{ env.arch }}.zip
path: .
- name: Extract packages
shell: pwsh
run: |
Expand-Archive -LiteralPath 'terraform-e2etest_${{ env.os }}_${{ env.arch }}.zip' -DestinationPath '.'
Expand-Archive -LiteralPath 'terraform_${{env.version}}_${{ env.os }}_${{ env.arch }}.zip' -DestinationPath '.'
- name: Run E2E Tests
env:
TF_ACC: 1
shell: cmd
run: |
e2etest.exe -test.v
build: Build docs source package as part of the release pipeline This should eventually grow to be a step that actually verifies the validity of the docs source prior to publishing the artifact that a downstream publishing pipeline can consume, but for the moment it's really just a placeholder since we have no such validation step and no downstream pipeline consuming this artifact. The general idea here is that the artifacts from this workflow should be sufficient for all downstream release steps to occur without any direct access to the Terraform CLI repository, and so this is intended to eventually meet that ideal but as of this commit the website docs publishing step _does_ still depend on direct access to this repository.
2022-01-03 19:03:00 +01:00
docs-source-package:
name: "Build documentation bundle"
runs-on: ubuntu-latest
needs:
- get-product-version
env:
version: ${{needs.get-product-version.outputs.product-version}}
steps:
- uses: actions/checkout@v2
# FIXME: We should include some sort of pre-validation step here, to
# confirm that the doc content is mechanically valid so that the
# publishing pipeline will be able to render all content without errors.
- name: "Create documentation source bundle"
run: |
(cd website && zip -9 -r ../terraform-cli-docs-source_${{ env.version }}.zip .)
- uses: actions/upload-artifact@v2
with:
name: terraform-cli-docs-source_${{ env.version }}.zip
path: terraform-cli-docs-source_${{ env.version }}.zip
if-no-files-found: error