provider/azurerm: Add example of encrypting a running linux (#14642)

* initial commit - 101-vm-from-user-image

* changed branch name

* not deploying - storage problems

* provisions vm but image not properly prepared

* storage not correct

* provisions properly

* changed main.tf to azuredeploy.tf

* added tfvars and info for README

* tfvars ignored and corrected file ext

* added CI config; added sane defaults for variables; updated deployment script, added mac specific deployment for local testing

* deploy.sh to be executable

* executable deploy files

* added CI files; changed vars

* prep for PR

* removal of old folder

* prep for PR

* wrong args for travis

* more PR prep

* updated README

* commented out variables in terraform.tfvars

* Topic 101 vm from user image (#2)

* initial commit - 101-vm-from-user-image
* added tfvars and info for README
* added CI config; added sane defaults for variables; updated deployment script, added mac specific deployment for local testing
* prep for PR

* added new template

* oops, left off master

* prep for PR

* correct repository for destination

* renamed scripts to be more intuitive; added check for docker

* merge vm simple; vm from image

* initial commit

* deploys locally

* updated deploy

* consolidated deploy and after_deploy into a single script; simplified ci process; added os_profile_linux_config

* added terraform show

* changed to allow http & https (like ARM tmplt)

* changed host_name & host_name variable desc

* added az cli check

* on this branch, only build test_dir; master will aggregate all the examples

* merge master

* added new constructs/naming for deploy scripts, etc.

* suppress az login output

* suppress az login output

* forgot about line breaks

* breaking build as an example

* fixing broken build example

* merge of CI config

* fixed grammar in readme

* prep for PR

* took out armviz button and minor README changes

* changed host_name

* fixed merge conflicts

* changed host_name variable

* updating Hashicorp's changes to merged simple linux branch

* updating files to merge w/master and prep for Hashicorp pr

* Revert "updating files to merge w/master and prep for Hashicorp pr"

This reverts commit b850cd5d2a858eff073fc5a1097a6813d0f8b362.

* Revert "updating Hashicorp's changes to merged simple linux branch"

This reverts commit dbaf8d14a9cdfcef0281919671357f6171ebd4e6.

* removing vm from user image example from this branch

* removed old branch

* azure-2-vms-loadbalancer-lbrules (#13)

* initial commit

* need to change lb_rule & nic

* deploys locally

* updated README

* updated travis and deploy scripts for Hari's repo

* renamed deploy script

* clean up

* prep for PR

* updated readme

* fixing conflict in .travis.yml

* initial commit; in progress

* in progress

* in progress; encryption fails

* in progress

* deploys successfully locally

* clean up; deploy typo fixed

* merging hashi master into this branch

* troubleshooting deploy

* added missing vars to deploy script

* updated README, outputs, and added graph

* simplified outputs

* reverting to Hashicorp's .travis.yml

* clean up

* readme graph correction

examples/azure-encrypt-running-linux-vm/README.md

@@ -0,0 +1,44 @@
+# Enable encryption on a running Linux VM. 
+
+This Terraform template was based on [this](https://github.com/Azure/azure-quickstart-templates/tree/master/201-encrypt-running-linux-vm) Azure Quickstart Template. Changes to the ARM template that may have occurred since the creation of this example may not be reflected in this Terraform template.
+
+This template enables encryption on a running linux vm using AAD client secret. This template assumes that the VM is located in the same region as the resource group. If not, please edit the template to pass appropriate location for the VM sub-resources.
+
+## Prerequisites:
+Azure Disk Encryption securely stores the encryption secrets in a specified Azure Key Vault.
+
+Create the Key Vault and assign appropriate access policies. You may use this script to ensure that your vault is properly configured: [AzureDiskEncryptionPreRequisiteSetup.ps1](https://github.com/Azure/azure-powershell/blob/10fc37e9141af3fde6f6f79b9d46339b73cf847d/src/ResourceManager/Compute/Commands.Compute/Extension/AzureDiskEncryption/Scripts/AzureDiskEncryptionPreRequisiteSetup.ps1)
+
+Use the below PS cmdlet for getting the `key_vault_secret_url` and `key_vault_resource_id`.
+
+```
+    Get-AzureRmKeyVault -VaultName $KeyVaultName -ResourceGroupName $rgname
+```
+
+References:
+
+- [White paper](https://azure.microsoft.com/en-us/documentation/articles/azure-security-disk-encryption/)
+- [Explore Azure Disk Encryption with Azure Powershell](https://blogs.msdn.microsoft.com/azuresecurity/2015/11/16/explore-azure-disk-encryption-with-azure-powershell/)
+- [Explore Azure Disk Encryption with Azure PowerShell – Part 2](http://blogs.msdn.com/b/azuresecurity/archive/2015/11/21/explore-azure-disk-encryption-with-azure-powershell-part-2.aspx)
+
+
+## main.tf
+The `main.tf` file contains the actual resources that will be deployed. It also contains the Azure Resource Group definition and any defined variables. 
+
+## outputs.tf
+This data is outputted when `terraform apply` is called, and can be queried using the `terraform output` command.
+
+## provider.tf
+You may leave the provider block in the `main.tf`, as it is in this template, or you can create a file called `provider.tf` and add it to your `.gitignore` file.
+
+Azure requires that an application is added to Azure Active Directory to generate the `client_id`, `client_secret`, and `tenant_id` needed by Terraform (`subscription_id` can be recovered from your Azure account details). Please go [here](https://www.terraform.io/docs/providers/azurerm/) for full instructions on how to create this to populate your `provider.tf` file.
+
+## terraform.tfvars
+If a `terraform.tfvars` file is present in the current directory, Terraform automatically loads it to populate variables. We don't recommend saving usernames and password to version control, but you can create a local secret variables file and use `-var-file` to load it.
+
+If you are committing this template to source control, please insure that you add this file to your .gitignore file.
+
+## variables.tf
+The `variables.tf` file contains all of the input parameters that the user can specify when deploying this Terraform template.
+
+![graph](/examples/azure-encrypt-running-linux-vm/graph.png)

examples/azure-encrypt-running-linux-vm/deploy.ci.sh

@@ -0,0 +1,60 @@
+#!/bin/bash
+
+set -o errexit -o nounset
+
+docker run --rm -it \
+  -e ARM_CLIENT_ID \
+  -e ARM_CLIENT_SECRET \
+  -e ARM_SUBSCRIPTION_ID \
+  -e ARM_TENANT_ID \
+  -e AAD_CLIENT_ID \
+  -e AAD_CLIENT_SECRET \
+  -e KEY_ENCRYPTION_KEY_URL \
+  -e KEY_VAULT_RESOURCE_ID \
+  -v $(pwd):/data \
+  --workdir=/data \
+  --entrypoint "/bin/sh" \
+  hashicorp/terraform:light \
+  -c "/bin/terraform get; \
+      /bin/terraform validate; \
+      /bin/terraform plan -out=out.tfplan \
+        -var resource_group=$KEY \
+        -var hostname=$KEY \
+        -var admin_username=$KEY \
+        -var admin_password=$PASSWORD \
+        -var passphrase=$PASSWORD \
+        -var key_vault_name=$KEY_VAULT_NAME \
+        -var aad_client_id=$AAD_CLIENT_ID \
+        -var aad_client_secret=$AAD_CLIENT_SECRET \
+        -var key_encryption_key_url=$KEY_ENCRYPTION_KEY_URL \
+        -var key_vault_resource_id=$KEY_VAULT_RESOURCE_ID; \
+      /bin/terraform apply out.tfplan"
+
+# cleanup deployed azure resources via azure-cli
+docker run --rm -it \
+  azuresdk/azure-cli-python \
+  sh -c "az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID > /dev/null; \
+         az vm show -g $KEY -n $KEY; \
+         az vm encryption show -g $KEY -n $KEY"
+
+# cleanup deployed azure resources via terraform
+docker run --rm -it \
+  -e ARM_CLIENT_ID \
+  -e ARM_CLIENT_SECRET \
+  -e ARM_SUBSCRIPTION_ID \
+  -e ARM_TENANT_ID \
+  -v $(pwd):/data \
+  --workdir=/data \
+  --entrypoint "/bin/sh" \
+  hashicorp/terraform:light \
+  -c "/bin/terraform destroy -force \
+    -var resource_group=$KEY \
+    -var hostname=$KEY \
+    -var admin_username=$KEY \
+    -var admin_password=$PASSWORD \
+    -var passphrase=$PASSWORD \
+    -var key_vault_name=$KEY_VAULT_NAME \
+    -var aad_client_id=$AAD_CLIENT_ID \
+    -var aad_client_secret=$AAD_CLIENT_SECRET \
+    -var key_encryption_key_url=$KEY_ENCRYPTION_KEY_URL \
+    -var key_vault_resource_id=$KEY_VAULT_RESOURCE_ID;"

examples/azure-encrypt-running-linux-vm/deploy.mac.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+set -o errexit -o nounset
+
+if docker -v; then
+
+  # generate a unique string for CI deployment
+  export KEY=$(cat /dev/urandom | env LC_CTYPE=C tr -cd 'a-z' | head -c 12)
+  export PASSWORD=$KEY$(cat /dev/urandom | env LC_CTYPE=C tr -cd 'A-Z' | head -c 2)$(cat /dev/urandom | env LC_CTYPE=C tr -cd '0-9' | head -c 2)
+  export EXISTING_RESOURCE_GROUP=permanent
+  export KEY_VAULT_NAME=permanentkeyvault
+
+/bin/sh ./deploy.ci.sh
+
+else
+  echo "Docker is used to run terraform commands, please install before run:  https://docs.docker.com/docker-for-mac/install/"
+fi

examples/azure-encrypt-running-linux-vm/main.tf

@@ -0,0 +1,223 @@
+# provider "azurerm" {
+#   subscription_id = "REPLACE-WITH-YOUR-SUBSCRIPTION-ID"
+#   client_id       = "REPLACE-WITH-YOUR-CLIENT-ID"
+#   client_secret   = "REPLACE-WITH-YOUR-CLIENT-SECRET"
+#   tenant_id       = "REPLACE-WITH-YOUR-TENANT-ID"
+# }
+
+resource "azurerm_resource_group" "rg" {
+  name     = "${var.resource_group}"
+  location = "${var.location}"
+}
+
+resource "azurerm_virtual_network" "vnet" {
+  name                = "${var.hostname}vnet"
+  location            = "${var.location}"
+  address_space       = ["${var.address_space}"]
+  resource_group_name = "${azurerm_resource_group.rg.name}"
+}
+
+resource "azurerm_subnet" "subnet" {
+  name                 = "${var.hostname}subnet"
+  virtual_network_name = "${azurerm_virtual_network.vnet.name}"
+  resource_group_name  = "${azurerm_resource_group.rg.name}"
+  address_prefix       = "${var.subnet_prefix}"
+}
+
+resource "azurerm_network_interface" "nic" {
+  name                = "nic"
+  location            = "${var.location}"
+  resource_group_name = "${azurerm_resource_group.rg.name}"
+
+  ip_configuration {
+    name                          = "ipconfig"
+    subnet_id                     = "${azurerm_subnet.subnet.id}"
+    private_ip_address_allocation = "Dynamic"
+  }
+}
+
+resource "azurerm_storage_account" "stor" {
+  name                = "${var.hostname}stor"
+  resource_group_name = "${azurerm_resource_group.rg.name}"
+  location            = "${azurerm_resource_group.rg.location}"
+  account_type        = "${var.storage_account_type}"
+}
+
+resource "azurerm_virtual_machine" "vm" {
+  name                  = "${var.hostname}"
+  location              = "${var.location}"
+  resource_group_name   = "${azurerm_resource_group.rg.name}"
+  vm_size               = "${var.vm_size}"
+  network_interface_ids = ["${azurerm_network_interface.nic.id}"]
+
+  storage_image_reference {
+    publisher = "${var.image_publisher}"
+    offer     = "${var.image_offer}"
+    sku       = "${var.image_sku}"
+    version   = "${var.image_version}"
+  }
+
+  storage_os_disk {
+    name          = "${var.hostname}osdisk"
+    create_option = "FromImage"
+    disk_size_gb  = "15"
+  }
+
+  os_profile {
+    computer_name  = "${var.hostname}"
+    admin_username = "${var.admin_username}"
+    admin_password = "${var.admin_password}"
+  }
+
+  os_profile_linux_config {
+    disable_password_authentication = false
+  }
+}
+
+resource "azurerm_template_deployment" "linux_vm" {
+  name                = "encrypt"
+  resource_group_name = "${azurerm_resource_group.rg.name}"
+  deployment_mode     = "Incremental"
+  depends_on          = ["azurerm_virtual_machine.vm"]
+
+  template_body = <<DEPLOY
+{
+  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
+  "contentVersion": "1.0.0.0",
+  "parameters": {
+    "aadClientID": {
+      "defaultValue": "${var.aad_client_id}",
+      "type": "string"
+    },
+    "aadClientSecret": {
+      "defaultValue": "${var.aad_client_secret}",
+      "type": "string"
+    },
+    "diskFormatQuery": {
+      "defaultValue": "",
+      "type": "string"
+    },
+    "encryptionOperation": {
+      "allowedValues": [ "EnableEncryption", "EnableEncryptionFormat" ],
+      "defaultValue": "${var.encryption_operation}",
+      "type": "string"
+    },
+    "volumeType": {
+      "allowedValues": [ "OS", "Data", "All" ],
+      "defaultValue": "${var.volume_type}",
+      "type": "string"
+    },
+    "keyEncryptionKeyURL": {
+      "defaultValue": "${var.key_encryption_key_url}",
+      "type": "string"
+    },
+    "keyVaultName": {
+      "defaultValue": "${var.key_vault_name}",
+      "type": "string"
+    },
+    "keyVaultResourceGroup": {
+      "defaultValue": "${azurerm_resource_group.rg.name}",
+      "type": "string"
+    },
+    "passphrase": {
+      "defaultValue": "${var.passphrase}",
+      "type": "string"
+    },
+    "sequenceVersion": {
+      "defaultValue": "${var.sequence_version}",
+      "type": "string"
+    },
+    "useKek": {
+      "allowedValues": [
+        "nokek",
+        "kek"
+      ],
+      "defaultValue": "${var.use_kek}",
+      "type": "string"
+    },
+    "vmName": {
+      "defaultValue": "${azurerm_virtual_machine.vm.name}",
+      "type": "string"
+    },
+    "_artifactsLocation": {
+      "type": "string",
+      "defaultValue": "${var.artifacts_location}"
+    },
+    "_artifactsLocationSasToken": {
+      "type": "string",
+      "defaultValue": "${var.artifacts_location_sas_token}"
+    }    
+  },
+  "variables": {
+    "extensionName": "${var.extension_name}",
+    "extensionVersion": "0.1",
+    "keyEncryptionAlgorithm": "RSA-OAEP",
+    "keyVaultURL": "https://${var.key_vault_name}.vault.azure.net/",
+    "keyVaultResourceID": "${var.key_vault_resource_id}",
+    "updateVmUrl": "${var.artifacts_location}/201-encrypt-running-linux-vm/updatevm-${var.use_kek}.json${var.artifacts_location_sas_token}"
+  },
+  "resources": [
+    {
+      "type": "Microsoft.Compute/virtualMachines/extensions",
+      "name": "[concat(parameters('vmName'),'/', variables('extensionName'))]",
+      "apiVersion": "2015-06-15",
+      "location": "[resourceGroup().location]",
+      "properties": {
+        "protectedSettings": {
+          "AADClientSecret": "[parameters('aadClientSecret')]",
+          "Passphrase": "[parameters('passphrase')]"
+        },
+        "publisher": "Microsoft.Azure.Security",
+        "settings": {
+          "AADClientID": "[parameters('aadClientID')]",
+          "DiskFormatQuery": "[parameters('diskFormatQuery')]",
+          "EncryptionOperation": "[parameters('encryptionOperation')]",
+          "KeyEncryptionAlgorithm": "[variables('keyEncryptionAlgorithm')]",
+          "KeyEncryptionKeyURL": "[parameters('keyEncryptionKeyURL')]",
+          "KeyVaultURL": "[variables('keyVaultURL')]",
+          "SequenceVersion": "[parameters('sequenceVersion')]",
+          "VolumeType": "[parameters('volumeType')]"
+        },
+        "type": "AzureDiskEncryptionForLinux",
+        "typeHandlerVersion": "[variables('extensionVersion')]"
+      }
+    },
+    {
+      "apiVersion": "2015-01-01",
+      "dependsOn": [
+        "[resourceId('Microsoft.Compute/virtualMachines/extensions',  parameters('vmName'), variables('extensionName'))]"
+      ],
+      "name": "[concat(parameters('vmName'), 'updateVm')]",
+      "type": "Microsoft.Resources/deployments",
+      "properties": {
+        "mode": "Incremental",
+        "parameters": {
+          "keyEncryptionKeyURL": {
+            "value": "[parameters('keyEncryptionKeyURL')]"
+          },
+          "keyVaultResourceID": {
+            "value": "[variables('keyVaultResourceID')]"
+          },
+          "keyVaultSecretUrl": {
+            "value": "[reference(resourceId('Microsoft.Compute/virtualMachines/extensions',  parameters('vmName'), variables('extensionName'))).instanceView.statuses[0].message]"
+          },
+          "vmName": {
+            "value": "[parameters('vmName')]"
+          }
+        },
+        "templateLink": {
+          "contentVersion": "1.0.0.0",
+          "uri": "[variables('updateVmUrl')]"
+        }
+      }
+    }
+  ],
+  "outputs": {
+    "BitLockerKey": {
+      "type": "string",
+      "value": "[reference(resourceId('Microsoft.Compute/virtualMachines/extensions',  parameters('vmName'), variables('extensionName'))).instanceView.statuses[0].message]"
+    }
+  }
+}
+DEPLOY
+}

examples/azure-encrypt-running-linux-vm/outputs.tf

@@ -0,0 +1,8 @@
+output "hostname" {
+  value = "${var.hostname}"
+}
+
+output "BitLockerKey" {
+  value     = "${azurerm_template_deployment.linux_vm.outputs["BitLockerKey"]}"
+  sensitive = true
+}

examples/azure-encrypt-running-linux-vm/variables.tf

@@ -0,0 +1,125 @@
+variable "resource_group" {
+  description = "Resource group name into which your new virtual machine will go."
+}
+
+variable "location" {
+  description = "The location/region where the virtual network is created. Changing this forces a new resource to be created."
+  default     = "southcentralus"
+}
+
+variable "hostname" {
+  description = "Used to form various names including the key vault, vm, and storage. Must be unique."
+}
+
+variable "address_space" {
+  description = "The address space that is used by the virtual network. You can supply more than one address space. Changing this forces a new resource to be created."
+  default     = "10.0.0.0/24"
+}
+
+variable "subnet_prefix" {
+  description = "The address prefix to use for the subnet."
+  default     = "10.0.0.0/24"
+}
+
+variable "storage_account_type" {
+  description = "Defines the type of storage account to be created. Valid options are Standard_LRS, Standard_ZRS, Standard_GRS, Standard_RAGRS, Premium_LRS. Changing this is sometimes valid - see the Azure documentation for more information on which types of accounts can be converted into other types."
+  default     = "Standard_LRS"
+}
+
+variable "vm_size" {
+  description = "Specifies the size of the virtual machine. This must be the same as the vm image from which you are copying."
+  default     = "Standard_A0"
+}
+
+variable "image_publisher" {
+  description = "name of the publisher of the image (az vm image list)"
+  default     = "Canonical"
+}
+
+variable "image_offer" {
+  description = "the name of the offer (az vm image list)"
+  default     = "UbuntuServer"
+}
+
+variable "image_sku" {
+  description = "image sku to apply (az vm image list)"
+  default     = "16.04-LTS"
+}
+
+variable "image_version" {
+  description = "version of the image to apply (az vm image list)"
+  default     = "latest"
+}
+
+variable "admin_username" {
+  description = "administrator user name for the vm"
+  default     = "vmadmin"
+}
+
+variable "admin_password" {
+  description = "administrator password for the vm (recommended to disable password auth)"
+}
+
+variable "aad_client_id" {
+  description = "Client ID of AAD app which has permissions to KeyVault"
+}
+
+variable "aad_client_secret" {
+  description = "Client Secret of AAD app which has permissions to KeyVault"
+}
+
+variable "disk_format_query" {
+  description = "The query string used to identify the disks to format and encrypt. This parameter only works when you set the EncryptionOperation as EnableEncryptionFormat. For example, passing [{\"dev_path\":\"/dev/md0\",\"name\":\"encryptedraid\",\"file_system\":\"ext4\"}] will format /dev/md0, encrypt it and mount it at /mnt/dataraid. This parameter should only be used for RAID devices. The specified device must not have any existing filesystem on it."
+  default     = ""
+}
+
+variable "encryption_operation" {
+  description = "EnableEncryption would encrypt the disks in place and EnableEncryptionFormat would format the disks directly"
+  default     = "EnableEncryption"
+}
+
+variable "volume_type" {
+  description = "Defines which drives should be encrypted. OS encryption is supported on RHEL 7.2, CentOS 7.2 & Ubuntu 16.04. Allowed values: OS, Data, All"
+  default     = "All"
+}
+
+variable "key_encryption_key_url" {
+  description = "URL of the KeyEncryptionKey used to encrypt the volume encryption key"
+}
+
+variable "key_vault_resource_id" {
+  description = "uri of Azure key vault resource"
+}
+
+variable "key_vault_name" {
+  description = "name of Azure key vault resource"
+}
+
+variable "passphrase" {
+  description = "The passphrase for the disks"
+}
+
+variable "extension_name" {
+  description = "the name of the vm extension"
+  default     = "AzureDiskEncryptionForLinux"
+}
+
+variable "sequence_version" {
+  description = "sequence version of the bitlocker operation. Increment this everytime an operation is performed on the same VM"
+  default     = 1
+}
+
+variable "use_kek" {
+  description = "Select kek if the secret should be encrypted with a key encryption key. Allowed values: kek, nokek"
+  default     = "kek"
+}
+
+variable "artifacts_location" {
+  description = "The base URI where artifacts required by this template are located. When the template is deployed using the accompanying scripts, a private location in the subscription will be used and this value will be automatically generated."
+  default     = "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master"
+}
+
+variable "artifacts_location_sas_token" {
+  description = "The sasToken required to access _artifactsLocation.  When the template is deployed using the accompanying scripts, a sasToken will be automatically generated."
+  default     = ""
+}