A Step-by-Step Guide: Importing Cisco ACI Nodes into IaC Using Python

This Tech Insight will provide practical examples of integrating existing Cisco Nexus 9k switches in an ACI fabric into a GIT/IaC environment using Python, with configuration templates in Terraform and Ansible.

Nearly a decade ago, I began using code to automate much of my daily tasks within my network engineering role. From my experience at AUTOCON0, the primary request has been for more practical examples of “How To”… and this post aims to provide exactly that.

Recently, I worked with a customer on modernizing their data center. One of our objectives was to make significant changes to their ACI environment, which had been manually configured using the NXOS-Style CLI and occasionally the GUI. Neither method aligns with best practices. As I took over this fabric to implement the changes, I decided to demonstrate how to integrate these brownfield elements into your GIT/IaC tooling environment, establishing a Single Source of Truth. In this post, I will show you how to import the existing registered Cisco Nexus 9k switches in the Cisco ACI fabric into the GIT/IaC tooling environment using Python.

I will provide examples of how to template the configuration in both Terraform and Ansible. Choose your own IaC tool… or be bold and run a database with “imperial code” (Python/Go).

The Challenge

While managing Cisco ACI node registration manually is not always the most cumbersome process, there are many additional steps associated with establishing a new node beyond simply registering it with the fabric. Therefore, this is the best place to start our journey in learning to automate our ACI fabric, as it is a dependency for all other steps in establishing a new node.

I wanted to develop a solution that would authenticate with the Cisco ACI fabric, fetch all registered nodes, store this data, and use it to automate node management through Infrastructure as Code (IaC) tools like Terraform and Ansible. The goal was to avoid the need for manually writing HCL or YAML code for potentially hundreds or even thousands of ACI nodes. The best method that I’ve had success with is using Python to populate Jinja2 templates of the HCL and YAML files.

I started by setting up my environment with the necessary libraries. Using Python, I installed requests for making HTTP requests, Jinja2 for templating, and a few other essential libraries:

import os
import requests
import json
import re
from datetime import datetime
from jinja2 import Environment, FileSystemLoader
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

• import os: I use this library to interact with the operating system, such as reading environment variables.
• import requests: This powerful library allows me to make HTTP requests to the Cisco ACI API.
• import json: I utilize this library to parse JSON responses from the API.
• import re: This regular expression library is invaluable for parsing strings, such as extracting pod IDs from device names.
• from datetime import datetime: This helps me handle date and time, crucial for timestamping my JSON files.
• from jinja2 import Environment, FileSystemLoader: I use Jinja2 for creating and managing templates, which is essential for generating Terraform and Ansible configurations.
• from requests.packages.urllib3.exceptions import InsecureRequestWarning: I need this to manage warnings related to insecure HTTP requests.
• requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

The line above disables annoying warnings about making insecure HTTP requests, because nearly every Cisco ACI Fabric uses a self-signed certificate. I’ve come across very few that actually take the steps to put a CA-signed certificate on their fabrics. While not “recommended” for production environments, it simplifies development by keeping the output clean.

Authenticating to the Cisco ACI Fabric:

The first step was to authenticate with the Cisco ACI fabric. I wrote a Python function to handle the authentication process, using environment variables to securely manage credentials. You never want to hard-code or store in configuration files your credentials; It is a very quick way to have your environment breached, and it can be nearly impossible to root out of GIT. While other more popular “production” methods of secret management might exist, such as HashiCorp Vault; they can be much more costly and complex to setup than the aims of this post. Now to execute this first step, in python, I wrote a function. A python function is a block of reusable code that performs a specific task. Functions help modularize and organize our python code into smaller, manageable pieces, making the code easier to understand.

Here’s the function I created to handle the authentication:

def get_token(TARGET_APIC, TARGET_USERNAME, TARGET_PASSWORD):
    url = f"{TARGET_APIC}/api/aaaLogin.json"
    payload = {
        "aaaUser": {
            "attributes": {
                "name": TARGET_USERNAME,
                "pwd": TARGET_PASSWORD
            }
        }
    }
    session = requests.Session()
    response = session.post(url, json=payload, verify=False)
    if response.status_code == 200:
        return session
    else:
        raise Exception(f"Failed logging into {TARGET_APIC} as {TARGET_USERNAME}")

• def get_token(TARGET_APIC, TARGET_USERNAME, TARGET_PASSWORD): This line defines a function named get_token that takes three parameters: the Cisco ACI APIC API endpoint (TARGET_APIC), the username (TARGET_USERNAME), and the password (TARGET_PASSWORD).
• url = f”{TARGET_APIC}/api/aaaLogin.json”: I construct the URL for the authentication endpoint by combining the Cisco ACI APIC API endpoint with the authentication path.
• payload = {…}: This dictionary contains the credentials required for authentication, structured in a way that the Cisco ACI APIC API expects.
• session = requests.Session(): I create a session object to persist parameters across multiple requests, improving performance and allowing for connection pooling.
• response = session.post(url, json=payload, verify=False): I send a POST request to the authentication endpoint with the credentials in JSON format. The verify=False parameter is used to skip SSL certificate verification for simplicity.
• if response.status_code == 200: I check if the response status code is 200, indicating a successful authentication.
• return session: If authentication is successful, the function returns the session object, which can be used for subsequent requests.
• else: raise Exception(…): If authentication fails, the function raises an exception with an error message.

Fetching Fabric Nodes:

Next, I needed to fetch the list of registered fabric nodes from the Cisco ACI APIC API. Again, I wrote a dedicated function that sends a HTTP GET request to the Cisco ACI APIC API and retrieves the configuration for the registration of ACI Nodes in JSON format:

def get_fabric_nodes(session, TARGET_APIC):
    url = f"{TARGET_APIC}/api/node/class/fabricNode.json"
    response = session.get(url, verify=False)
    if response.status_code == 200:
        response_json = response.json()
        timestamp = datetime.now().strftime('%Y%m%d%H%M%S')
        json_file_path = os.path.join('./import-json', f"export-fabric-nodes-{timestamp}.json")
        if not os.path.exists('./import-json'):
            os.makedirs('./import-json')
        with open(json_file_path, 'w') as json_file:
            json.dump(response_json, json_file, indent=2)
        return response_json
    else:
        raise Exception(f"Failed to query {TARGET_APIC}, status code: {response.status_code}")

• def get_fabric_nodes(session, TARGET_APIC): This line defines a function named get_fabric_nodes that takes two parameters:
  — The session object (session) created during authentication
  — The ACI API endpoint (TARGET_APIC).
• url = f”{TARGET_APIC}/api/node/class/fabricNode.json”: I construct the URL for fetching fabric nodes by combining the ACI API endpoint with the path to the fabric nodes data.
• response = session.get(url, verify=False): I send a GET request to the constructed URL using the session object. The verify=False parameter skips SSL certificate verification for simplicity.
• if response.status_code == 200: I check if the response status code is 200, indicating a successful request.
• response_json = response.json(): If the request is successful, I parse the JSON response containing the fabric nodes data.
• timestamp = datetime.now().strftime(‘%Y%m%d%H%M%S’): I generate a timestamp to uniquely identify the JSON file.
• json_file_path = os.path.join(‘./import-json’, f”export-fabric-nodes-{timestamp}.json”): I construct the file path for storing the JSON data, including the timestamp for uniqueness.
• if not os.path.exists(‘./import-json’): I check if the directory for storing the JSON file exists.
• os.makedirs(‘./import-json’): If the directory does not exist, I create it.
• with open(json_file_path, ‘w’) as json_file: I open the JSON file in write mode.
• json.dump(response_json, json_file, indent=2): I write the JSON data to the file with an indentation of 2 spaces for readability.
• return response_json: After storing the data, I return the JSON response so it can be used for further processing.
• else: raise Exception(…): If the request fails, I raise an exception with an error message indicating the failure.

What’s Jinja2 and It’s Templating Powers:

With the node data in hand, I used Jinja2 to create templates for both Terraform and Ansible using the examples from each tool’s respective website’s documentation as a baseline. Jinja2 is a powerful templating engine for Python. It allows you to create dynamic templates that can be rendered with data to produce formatted text. I first picked up this skill in formatting HTML files for micro-framework websites using python. However, I quickly and naturally start using it too format other text files of YAML and HCL to populate the variables in the reusability of those templates of code. The placeholders (enclosed in {{ }}) are our variables and will be replaced with actual data when the template is rendered.

Terraform Resource Template:

resource "aci_fabric_node_member" "{{ node.attributes.name }}" {
  name        = "{{ node.attributes.name }}"
  serial      = "{{ node.attributes.serial }}"
  annotation  = "orchestrator:terraform"
  node_id     = "{{ node.attributes.id }}"
  node_type   = "{{ node.attributes.nodeType }}"
  pod_id      = "{{ pod_id }}"
  role        = "{{ node.attributes.role }}"
  lifecycle {ignore_changes = all}
}

Terraform Import Command Template:

terraform import aci_fabric_node_member.{{ node.attributes.name }} "{{ node.attributes.dn }}"

Ansible Playbook Template:

---
- name: Add fabric node "{{ node.attributes.name }}"
  cisco.aci.aci_fabric_node:
    host: "{% raw %}{{ lookup('ansible.builtin.env', 'TF_VAR_CISCO_ACI_APIC_IP_ADDRESS') }}{% endraw %}"
    username: "{% raw %}{{ lookup('ansible.builtin.env', 'TF_VAR_CISCO_ACI_TERRAFORM_USERNAME') }}{% endraw %}"
    password: "{% raw %}{{ lookup('ansible.builtin.env', 'TF_VAR_CISCO_ACI_TERRAFORM_PASSWORD') }}{% endraw %}"
    serial: "{{ node.attributes.serial }}"
    node_id: "{{ node.attributes.id }}"
    switch: "{{ node.attributes.name }}"
    role: "{{ node.attributes.role }}"
    pod_id: "{{ pod_id }}"
    state: present
    annotation: "orchestrator:ansible"
    validate_certs: False
  delegate_to: localhost

Rendering the J2 Templates:

I wrote more python functions for each template to render the template using the fetched node data. These functions generate the necessary Terraform configuration files, Terraform import commands, and Ansible playbooks.

Terraform Resource Template Rendering:

The render_terraform_template function generates Terraform configuration files for the fabric nodes:

def render_terraform_template(json_data, template_file, output_dir):
    env = Environment(loader=FileSystemLoader(searchpath=’./terraform/templates’))
    try:
        template = env.get_template(template_file)
    except Exception as e:
        print(f”Error loading template {template_file}: {e}”)
        raise

    if not os.path.exists(output_dir):
        os.makedirs(output_dir)

    timestamp = datetime.now().strftime(‘%Y%m%d%H%M%S’)
    tf_file_path = os.path.join(output_dir, f”3-import_fabric_nodes_{timestamp}.tf”)

    try:
        with open(tf_file_path, ‘w’) as f:
            for node in json_data[“imdata”]:
                dn = node[“fabricNode”][“attributes”][“dn”]
                pod_id = extract_pod_id(dn)
                terraform_resource = template.render(node=node[“fabricNode”], pod_id=pod_id)
                f.write(terraform_resource + ‘\n’)
            print(f”Terraform configuration written to: {tf_file_path}”)
    except Exception as e:
        print(f”Failed to write Terraform configuration to file: {e}”)
        raise

• env = Environment(loader=FileSystemLoader(searchpath=’./terraform/templates’)): I create a Jinja2 environment, specifying the directory where the Terraform templates are located.
• template = env.get_template(template_file): I load the specified Terraform template file.
• if not os.path.exists(output_dir): I check if the output directory exists.
• os.makedirs(output_dir): If the output directory does not exist, I create it.
• timestamp = datetime.now().strftime(‘%Y%m%d%H%M%S’): I generate a timestamp to uniquely identify the output file.
• tf_file_path = os.path.join(output_dir, f”3-import_fabric_nodes_{timestamp}.tf”): I construct the file path for the output file, including the timestamp.
• with open(tf_file_path, ‘w’) as f: I open the output file in write mode.
• for node in json_data[“imdata”]: I iterate over each node in the JSON data.
• dn = node[“fabricNode”][“attributes”][“dn”]: I extract the distinguished name (DN) of the node.
• pod_id = extract_pod_id(dn): I extract the pod ID from the DN using a helper function.
• terraform_resource = template.render(node=node[“fabricNode”], pod_id=pod_id): I render the template with the node data and pod ID.
• f.write(terraform_resource + ‘\n’): I write the rendered content to the output file.
• print(f”Terraform configuration written to: {tf_file_path}”): I print a confirmation message indicating the file has been written successfully.

Terraform Import Commands Template Rendering:

The render_tf_cmd_template function generates Terraform import command files for the fabric nodes, and follows a similar structure to render_terraform_template, but it generates Terraform import commands instead of resource configurations. You have to generate the import commands, becuase these resources were not created from Terraform and do not exist in the statefile.

As a result, Terraform will attempt to create them as “net-new” and some APIs will provide push back stating the object is duplicated; Forcing the Terraform runtime to fail. Fortunately or Unfortunately, the Cisco ACI APIC API does not have this logic and push back; I caution you to get into this habit regardless and avoid any unforeseen mishaps with the Cisco ACI APIC API. Notice a primary difference in the template and the file extension (.txt instead of .tf).

def render_tf_cmd_template(json_data, template_file, output_dir):
    env = Environment(loader=FileSystemLoader(searchpath=’./terraform/templates’))
    try:
        template = env.get_template(template_file)
    except Exception as e:
        print(f”Error loading template {template_file}: {e}”)
        raise

    if not os.path.exists(output_dir):
        os.makedirs(output_dir)

    timestamp = datetime.now().strftime(‘%Y%m%d%H%M%S’)
    tf_file_path = os.path.join(output_dir, f”3-import_fabric_nodes_{timestamp}.txt”)

    try:
        with open(tf_file_path, ‘w’) as f:
            for node in json_data[“imdata”]:
                terraform_resource = template.render(node=node[“fabricNode”])
                f.write(terraform_resource + ‘\n’)
            print(f”Terraform commands written to: {tf_file_path}”)
    except Exception as e:
        print(f”Failed to write Terraform commands to file: {e}”)
        raise

Ansible Playbook Template Rendering:

The render_ansible_template function generates Ansible playbooks for the fabric nodes. Again, this python function for the Ansible formatting follows a similar structure to render_terraform_template, but it generates Ansible playbooks with the file extension .yaml.

def render_ansible_template(json_data, template_file, output_dir):
    env = Environment(loader=FileSystemLoader(searchpath=’./ansible/templates’))
    try:
        template = env.get_template(template_file)
    except Exception as e:
        print(f”Error loading template {template_file}: {e}”)
        raise

    if not os.path.exists(output_dir):
        os.makedirs(output_dir)

    timestamp = datetime.now().strftime(‘%Y%m%d%H%M%S’)
    ansible_file_path = os.path.join(output_dir, f”3-import_fabric_nodes_{timestamp}.yaml”)

    try:
        with open(ansible_file_path, ‘w’) as f:
            for node in json_data[“imdata”]:
                dn = node[“fabricNode”][“attributes”][“dn”]
                pod_id = extract_pod_id(dn)
                ansible_resource = template.render(node=node[“fabricNode”], pod_id=pod_id)
                f.write(ansible_resource + ‘\n’)
            print(f”Ansible configuration written to: {ansible_file_path}”)
    except Exception as e:
        print(f”Failed to write Ansible configuration to file: {e}”)
        raise

Triggering all the Functions:

Finally, I trigger the entire workflow by chaining these functions together in the main function as the entry point of the script. This approach ensures that the process, from authentication to template generation, is the declaration (or declarative) part of my workflow as it orchestrates the sequence of operations required to authenticate with the Cisco ACI fabric, fetch the registered nodes, and generate the necessary configuration files using templates. Here’s how my main function works:

def main():
    TARGET_APIC = os.getenv('TF_VAR_CISCO_ACI_APIC_IP_ADDRESS')
    TARGET_USERNAME = os.getenv('TF_VAR_CISCO_ACI_TERRAFORM_USERNAME')
    TARGET_PASSWORD = os.getenv('TF_VAR_CISCO_ACI_TERRAFORM_PASSWORD')
    session = get_token(TARGET_APIC, TARGET_USERNAME, TARGET_PASSWORD)
    fabric_nodes_json = get_fabric_nodes(session, TARGET_APIC)
    render_terraform_template(fabric_nodes_json, 'aci_fabric_node_member.j2', './terraform')
    render_tf_cmd_template(fabric_nodes_json, 'aci_fabric_node_member_tf_import_cmds.j2', './terraform/tf-import-commands')
    render_ansible_template(fabric_nodes_json, 'aci_fabric_node_member.j2', './ansible')

• TARGET_APIC = os.getenv(‘TF_VAR_CISCO_ACI_APIC_IP_ADDRESS’): This line retrieves the ACI API endpoint from the environment variables.
• TARGET_USERNAME = os.getenv(‘TF_VAR_CISCO_ACI_TERRAFORM_USERNAME’): This line retrieves the username from the environment variables.
• TARGET_PASSWORD = os.getenv(‘TF_VAR_CISCO_ACI_TERRAFORM_PASSWORD’): This line retrieves the password from the environment variables.
• session = get_token(TARGET_APIC, TARGET_USERNAME, TARGET_PASSWORD): The get_token function is called to authenticate with the Cisco ACI fabric using the provided credentials. It returns a session object that will be used for subsequent API requests.
• fabric_nodes_json = get_fabric_nodes(session, TARGET_APIC): The get_fabric_nodes function is called with the session object and ACI API endpoint to fetch the registered fabric nodes. The function returns the nodes in JSON format.
• render_terraform_template(fabric_nodes_json, ‘aci_fabric_node_member.j2’, ‘./terraform’): The render_terraform_template function is called to generate the Terraform configuration files using the fetched node data and the specified template.
• render_tf_cmd_template(fabric_nodes_json, ‘aci_fabric_node_member_tf_import_cmds.j2’, ‘./terraform/tf-import-commands’): The render_tf_cmd_template function is called to generate the Terraform import command files.
• render_ansible_template(fabric_nodes_json, ‘aci_fabric_node_member.j2’, ‘./ansible’): The render_ansible_template function is called to generate the Ansible playbooks.

Now, we’ve journeyed through the process of automating the registration and management of Cisco ACI nodes using Python to template both Terraform, and Ansible. Automation is not just for the so-called unicorn engineers. With practical examples and a clear understanding of the tools at your disposal, anyone can harness the power of code to simplify their workflows.