Terraform L100 ---------------------- Hands-on tasks for this workshop will be using Terraform against a PAN-OS firewall, from your command-line environment. For each task, you can use the copy button to take the command into the clipboard, but ensure you replace any UPPERCASE_VARIABLES with your own values. Assumptions: A host with the following installed: curl, git, `Terraform 1.x `_, and any prerequisite requirements for those items. Plus some form of text editor on your host. And, access to a PAN-OS next-generation firewall from your host. Note: * Lookout for text in uppercase, such as ``YOUR_FIREWALL_IP``, and replace the text with the relevant value for your environment. * Commands you could/should execute are denoted with ``$`` at the start, and have copy buttons; other text boxes show expected outputs. * You may wish to have a text editor open on your machine, to amend copied commands before pasting them into the host's CLI. Terraform pre-flight checks ================================================ First let's check that Terraform is installed on our host. .. code-block:: :class: copy-button $ terraform -version The output should look something like this, and may contain a warning if you are not on the very latest version: .. code-block:: Terraform v1.0.10 Your version of Terraform is out of date! The latest version is 1.1.0. You can update by downloading from https://www.terraform.io/downloads.html It is fine to ignore the warning, as long as your version is at least 1.0.0 or higher. Download Terraform configs ================================================ In this workshop, we're going to be using some existing Terraform configs. We'll use git to download them from a repository... .. code-block:: :class: copy-button $ git clone https://github.com/jamesholland-uk/automation-workshops.git ...then move into the Terraform directory .. code-block:: :class: copy-button $ cd automation-workshops/terraform In the Terraform directory, a number of files will be present: .. code-block:: :class: copy-button $ ls -l .. code-block:: 00.tf \ 01.tf.bak |- These are the files where we 02.tf.bak |- write our Terraform code 03.tf.bak / commit.sh - We will use this script to perform a commit after making changes providers.tf - This file describes the third party systems, such as PAN-OS, which we want to use variables.tf - This file defines the variables we are going to use in our Terraform code Setup the PAN-OS variables for Terraform ================================================ Terraform needs to know the details for the hosts against which it should execute. There are a variety of methods for this, today we will store the host, admin username, and admin password as variables in memory for the duration of the CLI session with the ``export`` command: .. code-block:: :class: copy-button $ export TF_VAR_panos_hostname="YOUR_FIREWALL_IP" $ export TF_VAR_panos_username="YOUR_ADMIN_USERNAME" $ export TF_VAR_panos_password="YOUR_ADMIN_PASSWORD" For example: .. code-block:: $ export TF_VAR_panos_hostname="10.10.10.10" $ export TF_VAR_panos_username="adminuser" $ export TF_VAR_panos_password="ChangeMe123!" In production environments, credentials should be stored, accessed and used securely, per the security policy and compliance requirements. Today, in this environment, we will use these credentials stored in variables for executing Terraform. Initialising Terraform ================================================ Terraform uses a multi-stage process, the first of which is to ``init``, short for `initialise `_. Run the following command, which sets up the directory Terraform works with (.terraform/), as well as downloading various required files ready to be used at runtime. .. code-block:: :class: copy-button $ terraform init The output should look something like this: .. code-block:: :emphasize-lines: 4,5,6,17 Initializing the backend... Initializing provider plugins... - Finding paloaltonetworks/panos versions matching "~> 1.8.3"... - Installing paloaltonetworks/panos v1.8.3... - Installed paloaltonetworks/panos v1.8.3 (signed by a HashiCorp partner, key ID D5D93F98EFA33E83) Partner and community providers are signed by their developers. If you'd like to know more about provider signing, you can read about it here: https://www.terraform.io/docs/plugins/signing.html Terraform has created a lock file .terraform.lock.hcl to record the provider selections it made above. Include this file in your version control repository so that Terraform can guarantee to make the same selections by default when you run "terraform init" in the future. Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. Some particularly interesting lines in the output are the first block of highlighted text, showing the download of the PAN-OS ``provider``. A provider is responsible for the interaction between Terraform and a remote system, in this case a PAN-OS firewalls. Providers are available for many other prducts and cloud services. Also of note is the second section of highlighted text, confirming Terraform has been successfully initialised. The Terraform Plan ================================================ The second stage when using Terraform is usually ``plan``, which as you may expect, asks Terraform to plan what it will do next. Given the resources you are asking Terraform to work with, which could be anything from cloud infrastructure to PAN-OS firewalls, Terraform will first assess the real state of those resources. Terraform will then compare that real state with your desired state, which is defined by Terraform configuration files (those files in the local directory with .tf extension), and show you the difference between the two. This therefore describes the alterations which Terraform would make the next time it is executed to ``apply`` changes, to change the real state into the desired state. Let's look at our current Terraform config file. Ignoring the providers.tf and variables.tf files for now, the remaining .tf file in scope right now is 00.tf, use this command to show that file's contents: .. code-block:: :class: copy-button $ cat 00.tf .. code-block:: resource "panos_address_object" "terraform-address-object-1" { name = "terraform-address-object-1" value = "192.168.80.1/32" description = "Address object 1 from Terraform" } Note that convention is to name the file with your configuration ``main.tf``, but it could be called anything. We have several exercises, so we are using ``00.tf``, ``01.tf``, ``02.tf``, etc The file 00.tf defines a single resource, an address object called ``terraform-address-object-1`` with value ``192.168.80.1/32``. By logging into your firewall's web GUI or CLI, you will see there are no address objects configured at present. Now, let's run the following command to ask Terraform to show the plan: .. code-block:: :class: copy-button $ terraform plan The output should look something like this: .. code-block:: Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # panos_address_object.terraform-address-object-1 will be created + resource "panos_address_object" "terraform-address-object-1" { + description = "Address object 1 from Terraform" + device_group = "shared" + id = (known after apply) + name = "terraform-address-object-1" + type = "ip-netmask" + value = "192.168.80.1/32" + vsys = "vsys1" } Plan: 1 to add, 0 to change, 0 to destroy. ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now. As expected, the plan (which remember is the difference between the real state and the desired state) will add a single address object. Applying Changes with Terraform ================================================ As previously mentioned, Terraform wants to use all .tf files in the local directory, so lets remove the first file from scope, and introduce the second file: .. code-block:: :class: copy-button $ mv 00.tf 00.tf.bak $ mv 01.tf.bak 01.tf The second Terraform file creates more address objects and a creates an address group: .. code-block:: :class: copy-button $ cat 01.tf .. code-block:: resource "panos_address_group" "terraform-address-group" { name = "terraform-address-group" description = "Group of internal hosts" static_addresses = [ panos_address_object.terraform-address-object-1.name, panos_address_object.terraform-address-object-2.name ] } resource "panos_address_object" "terraform-address-object-1" { name = "terraform-address-object-1" value = "192.168.80.1/32" description = "Address object 1 from Terraform" } resource "panos_address_object" "terraform-address-object-2" { name = "terraform-address-object-2" value = "192.168.80.2/32" description = "Address object 2 from Terraform" } resource "panos_address_object" "terraform-address-object-3" { name = "terraform-address-object-3" value = "192.168.80.3/32" description = "Address object 3 from Terraform" } Perform the ``terraform plan`` command to test run the changes: .. code-block:: :class: copy-button $ terraform plan The output should look something like this: .. code-block:: :emphasize-lines: 6, 17, 28, 39, 50 Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # panos_address_group.terraform-address-group will be created + resource "panos_address_group" "terraform-address-group" { + id = (known after apply) + name = "terraform-address-group" + static_addresses = [ + "terraform-address-object-1", + "terraform-address-object-2", ] + vsys = "vsys1" } # panos_address_object.terraform-address-object-1 will be created + resource "panos_address_object" "terraform-address-object-1" { + description = "Address object 1 from Terraform" + device_group = "shared" + id = (known after apply) + name = "terraform-address-object-1" + type = "ip-netmask" + value = "192.168.80.1/32" + vsys = "vsys1" } # panos_address_object.terraform-address-object-2 will be created + resource "panos_address_object" "terraform-address-object-2" { + description = "Address object 2 from Terraform" + device_group = "shared" + id = (known after apply) + name = "terraform-address-object-2" + type = "ip-netmask" + value = "192.168.80.2/32" + vsys = "vsys1" } # panos_address_object.terraform-address-object-3 will be created + resource "panos_address_object" "terraform-address-object-3" { + description = "Address object 3 from Terraform" + device_group = "shared" + id = (known after apply) + name = "terraform-address-object-3" + type = "ip-netmask" + value = "192.168.80.3/32" + vsys = "vsys1" } Plan: 4 to add, 0 to change, 0 to destroy. ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now. Each of the first four highlighted sections show a new object being created for our firewall. The final highlighted section gives a summary, telling us 4 new objects will be created, none to be modified, none to be destroyed. By logging into the firewall web GUI, you should be able to observe that these objects do not already exist. To make these changes on the firewall, we use the ``terraform apply`` command. It is performed like this: .. code-block:: :class: copy-button $ terraform apply The plan will be re-generated, and you will be asked for confirmation to make the changes, so type ``yes``: .. code-block:: :emphasize-lines: 5 Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes Our address objects and group are now created in the candidate configuration. This can be confirmed by observing the web GUI for the firewall. Note that in the ``plan`` output, as in the configuration file, the address group was listed first before the address objects. Terraform executes in the order which observes any dependencies, and hence during the ``apply`` operation, it created the address objects first, then added them to the group (as adding non-existent address objects to an address group would have failed in PAN-OS). Idempotence ====================================================== Note that at this point, you should be able to perform ``terraform apply -refresh-only``, and because Terraform and the PAN-OS provider perform idempotent operations, you will see that no changes need to be made. Idempotence calls for operations that can be applied multiple times without changing the result, and in this context, duplicate objects will not be created each time ``terraform apply`` is executed, the output should just end with: .. code-block:: No changes. Your infrastructure still matches the configuration. Terraform has checked that the real remote objects still match the result of your most recent changes, and found no differences. Committing and/or Pushing Changes with Terraform ====================================================== **IMPORTANT**: Terraform's methodology is to expect that when configuration changes are executed with the ``terraform plan`` command, each configuration item is made live straight away. PAN-OS works differently, where configuration can (and some times has to be) be built up across objects, rules, zones, interfaces and more, and the configuration is only valid once all the parts are in place. All the various parts of configuration are then made live with a ``commit`` operation. This difference in methodology between Terraform and PAN-OS requires `commits to be performed via a specific mechanism `_; there are a variety of approaches to performing PAN-OS commits with Terraform, today we will use a simple script: .. code-block:: :class: copy-button $ ./commit.sh $TF_VAR_panos_hostname $TF_VAR_panos_username $TF_VAR_panos_password The script will initiate a commit, and wait through the active (``ACT``) stage, until it is finished (``FIN``). .. code-block:: ./commit.sh $TF_VAR_panos_hostname $TF_VAR_panos_username $TF_VAR_panos_password Commit status: ACT Commit status: ACT Commit status: ACT Commit status: ACT Commit status: ACT Commit status: ACT Commit status: ACT Final commit status: FIN The changes are now live in the running configuration. Applying More Changes ================================================ Let's make some more changes. We will use the third Terraform file for this, so execute the commands below: .. code-block:: :class: copy-button $ mv 01.tf 01.tf.bak $ mv 02.tf.bak 02.tf You can view the next config file with this command: .. code-block:: :class: copy-button $ cat 02.tf The changes between the last config file and this file include: the absence of address-object-3, a description being added to the existing address group, and the addition of two zones and two security rules. Run ``terraform plan`` to see what changes Terraform is lining up: .. code-block:: :class: copy-button $ terraform plan Because this Terraform file includes only two of the objects previously created, one address object is listed for deletion; that object has been removed from the config file and hence been removed from the desired state in Terraform's eyes. This is important in understanding how Terraform deals with state, and mapping the desired state listed in the config file to the real world state. The output should be something like this (truncated for brevity): .. code-block:: :emphasize-lines: 4,5,6,9,10,14 Terraform will perform the following actions: . . # panos_address_group.terraform-address-group will be updated in-place ~ resource "panos_address_group" "terraform-address-group" { + description = "Group of internal hosts" . . # panos_address_object.terraform-address-object-3 will be destroyed # (because panos_address_object.terraform-address-object-3 is not in configuration) . . . Plan: 3 to add, 1 to change, 1 to destroy. Note the modification of the address group (adding a description), the removal of address-object-3, and the addition of two zones and some security rules. Make these changes to the firewall, using ``terraform apply`` command, but this time we can skip the confirmation prompt like this: .. code-block:: :class: copy-button $ terraform apply --auto-approve Finally, execute the commit script, and confirm the new zones and rules are live on the firewall's running configuration bia the web GUI: .. code-block:: :class: copy-button $ ./commit.sh $TF_VAR_panos_hostname $TF_VAR_panos_username $TF_VAR_panos_password Getting Information with Terraform ================================================ The next exercise will show that Terraform can gather PAN-OS state information when required. This uses a ``data source``, the part of a Terraform provider responsible for gathering data. Switch to this Terraform file, then observe the new lines in this Terraform file, with these commands: .. code-block:: :class: copy-button $ mv 02.tf 02.tf.bak $ mv 03.tf.bak 03.tf .. code-block:: :class: copy-button $ diff 02.tf.bak 03.tf The output from ``diff`` shows we have added some extra lines from the previous exercise: .. code-block:: :emphasize-lines: 4,6,7 --- > } > > data "panos_system_info" "ngfw_info" { } > > output "the_info" { > value = data.panos_system_info.ngfw_info > } We are using the `panos_system_info `_ data source to mimic a ``show system info`` on the CLI. The ``data`` line instructs Terraform to collect the data, and the ``output`` lines instruct Terraform to send the data out at the end of running tasks. Let's gather the data, by again using the ``plan`` command: .. code-block:: :class: copy-button $ terraform plan The output should look something like this (truncated for brevity): .. code-block:: Changes to Outputs: + the_info = { + id = "10.110.255.4" + info = { "app-version" = "8468-6979" "av-version" = "0" "cloud-mode" = "cloud" "default-gateway" = "10.110.255.1" "device-certificate-status" = "None" "device-dictionary-version" = "1-211" "devicename" = "lab-fw" "family" = "vm" . . . "wildfire-rt" = "Disabled" "wildfire-version" = "0" } + version_major = 10 + version_minor = 1 + version_patch = 3 } The first block is the equivalent output from the CLI command ``show system info``, followed by the PAN-OS version broken down by major, minor and patch version. This can be useful on its own, but the information could also be parsed and used in other areas of our Terraform logic, where we might choose to perform a tasks if the PAN-OS software is equal to or greater than a certain version number.