Ansible, Rackspace Cloud Servers, and Virtualenv

Ansible is a powerful tool for managing and configuring servers. It can also be used to create new cloud servers and then configure them automatically for us. For a system administrator, engineer, and developer, ansible can save us a lot of time and hassle by automating a lot of our routine tasks.

Here’s a tutorial guide for using ansible to build and configure new Rackspace Cloud servers.

I will walk through setting up a new python virtual environment, installing pyrax and ansible inside of of the virtualenv, and then write an ansible playbook and role for building a Rackspace Cloud server, setting up DNS entries for the new server, and installing our favorite packages on the new server.

You can find the complete source for this ansible Rackspace Cloud Servers example on my github: ansible-rackspace-servers-example

Set up your virtualenv with pyrax and ansible

First, let’s create a new python virtualenv for use with Rackspace Cloud. The new virtualenv contains everything needed to build new Rackspace Cloud servers using the cloud servers API.

Create a new python virtualenv for use with Rackspace Cloud

virtualenv rackspace

The output from the virtualenv command should look something like this:

nick@mbp: ~/virtualenvs$ virtualenv rackspace_cloud
New python executable in rackspace_cloud/bin/python
Installing setuptools, pip...done.

Once the virtualenv has been created, let’s activate it so we can work with it:

source rackspace_cloud/bin/activate

This is what it will look like once activated on my machine. Notice how the prompt has changed to include the virtualenv name “rackspace_cloud” I have just activated.

nick@mbp: ~/virtualenvs$ source rackspace_cloud/bin/activate
(rackspace_cloud)nick@mbp: ~/virtualenvs$

Install the pyrax python package for working with Rackspace Cloud

Next we’ll need to install the pyrax package which is needed to work with the Rackspace Cloud API in python. Installing pyrax will also install all of the other prerequisite packages needed by pyrax, such as python-novaclient, python-keystoneclient, rackspace-novaclient, and rackspace-auth-openstack.

pip install pyrax

Here’s what the output looks like on my workstation: https://gist.github.com/nicholaskuechler/603ee13bd74944866650

Install Ansible in to the virtualenv

Now that we have pyrax installed, let’s go ahead and install ansible in to our new virtualenv.

pip install ansible

Here’s the output from my workstation: https://gist.github.com/nicholaskuechler/8f4812226c6a908dbc9a

Configuration files for pyrax and ansible

Now that we have our python virtualenv set up and the pyrax and ansible packages installed, we need to create a couple configuration files for use with pyrax and ansible.

Configuration file for pyrax

We will place our Rackspace Cloud username and API key in the pyrax configuration, which pyrax will use when making API calls to Rackspace Cloud products.

Create a new file: ~/.rackspace_cloud_credentials

Here’s what my .rackspace_cloud_credentials pyrax config looks like:

[rackspace_cloud]
username = mycloudusername
api_key = 0123456789abcde

Configuration file for ansible

An ansible configuration file is not necessary as the defaults will work without any issues. There’s one setting in particular, though, which I find helpful when creating a lot of new Rackspace cloud servers via ansible. This setting is the SSH setting StrictHostKeyChecking=no, which means I don’t have to manually confirm a host key when trying to SSH to it. This is a real advantage during automation and playbook runs, since we may not want to have a human confirm the SSH connection.

The default ansible configuration file lives at: ~/.ansible.cfg

Here’s my .ansible.cfg with the option to disable SSH’s strict host key checking:

[ssh_connection]
ssh_args = -o StrictHostKeyChecking=no

Ansible playbook to create Rackspace Cloud Servers

Now that we’ve installed and configured virtualenv, pyrax, and ansible, we’re ready to write an ansible playbook for building new Rackspace cloud servers.

First let’s make a new directory for our ansible playbook where the inventory file, play books, and ansible roles will live:

mkdir ansible-rackspace-servers

Use a dynamic ansible inventory in virtualenv with Rackspace Cloud

Many people use static ansible inventory files when working with their servers. Since we’re using a cloud provider, let’s use a dynamic inventory. But there’s a catch due to installing ansible in a virtualenv: we need to specify the path to the virtualenv python, because ansible will default to the system python rather than the python installed in the virtualenv.

In the ansible-rackspace-servers directory we’ve just created, make a new file for the ansible virtualenv inventory file. I named my virtualenv inventory file: virtualenv-inventory.yml

Here are the contents of my ansible virtualenv inventory file: virtualenv-inventory.yml

[localhost]
localhost ansible_connection=local ansible_python_interpreter=/Users/nick/virtualenvs/rackspace_cloud/bin/python

The important configuration piece here is ansible_python_interpreter where we specify the full path to our “rackspace_cloud” virtualenv’s python binary. Without the ansible_python_interpreter setting, ansible will try to use the default system python which is probably /usr/bin/python, and as such it will not find our virtualenv packages like pyrax and ansible that we installed in our virtualenv named rackspace_cloud.

A basic playbook to create a new Rackspace Cloud server

Reading an ansible playbook is fairly straightforward, but writing them can be a little trickier. I’ve created a simple playbook you can use to build new cloud servers in the Rackspace Cloud.

Let’s create a new file for our playbook. I’ve called mine build-cloud-server.yml. Here’s my playbook to create a new Rackspace cloud instance: build-cloud-server.yml

---
- name: Create a Rackspace Cloud Server
  hosts: localhost
  user: root
  connection: local
  gather_facts: False

  vars:
   # this is the name we will see in the Rackspace Cloud control panel, and 
   # this will also be the hostname of our new server
   - name: admin.enhancedtest.net
   # the flavor specifies the server side of our instance
   - flavor: performance1-1
   # the image specifies the linux distro we will use for our server
   # note: this image UUID is for Ubuntu 14.10 PVHVM
   - image: 0766e5df-d60a-4100-ae8c-07f27ec0148f
   # the region is the Rackspace Cloud region we want to build our server in
   - region: DFW
   # credentials specifies the location of our pyrax configuration file we created earlier
   - credentials: /Users/nick/.rackspace_cloud_credentials
   # I like to drop in my SSH pub key automatically when I create the server
   # so that I can ssh in without a password
   # Note: Instead of dropping in a file, you can use a stored Rackspace key
   # when you build the server by editing key_name below to your key's name.
   - files:
        /root/.ssh/authorized_keys: /Users/nick/.ssh/id_rsa.pub

  tasks:
    - name: Rackspace cloud server build request
      local_action:
        module: rax
        credentials: "{{ credentials }}"
        name: "{{ name }}"
        flavor: "{{ flavor }}"
        image: "{{ image }}"
        region: "{{ region }}"
        # key_name - specifies the Rackspace cloud key to add to the server upon creation
        #key_name: my_rackspace_key
        files: "{{ files }}"
        # wait - specifies that we should wait until the server is fully created before proceeding
        wait: yes
        # state - present means we want our server to exist
        state: present
        # specify that we want both a public network (public IPv4) and
        # a private network (10. aka service net)
        networks:
          - private
          - public
        # group - specifies metadata to add to the new server with a server group
        #group: deploy
      # register is an ansible term to save the output in to a variable named rax
      register: rax

The ansible task to create a new Rackspace cloud server is pretty straightforward. It uses the rax module included by default with ansible. There are many different options we can work with when creating a new server, but I’ve included the important ones in this playbook.

Run the ansible playbook to create new Rackspace Cloud Server

Let’s run our new playbook and see what happens! To run the playbook, we need to call ansible-playbook along with our dynamic inventory file virtualenv-inventory.yml and our playbook yaml build-cloud-server.yml. Note: make sure you’re still in the virtualenv you’ve created for this project!

ansible-playbook -i virtualenv-inventory.yml build-cloud-server.yml -vvvv

Notes:

• -i virtualenv-inventory.yml specifies the inventory file to use. The “-i” denotes the inventory option.
•  -vvvv specifies that we want really verbose ansible-playbook output, so that we can debug and troubleshoot if something goes wrong.

Here’s the output when running the build-cloud-server.yml playbook. Note I have obfuscated some of the output as it contains personal information: https://gist.github.com/nicholaskuechler/85aa9e21998bea87a3c5

Success! We’ve just created a new Rackspace Cloud server using an ansible playbook inside of a virtualenv.

Kicking it up a notch: Create a server, add DNS records, and Install some packages

When creating a new server, there are other tasks I have to complete, such as adding new forward and reverse DNS entries and installing a set of base packages I like to use on my new servers.

We can do all of these tasks in ansible!

Use Ansible to create Rackspace Cloud DNS entries for a cloud server

Now that we have a play for building a new server, let’s add a couple tasks for creating DNS A and PTR records for the newly created server automatically. Another benefit is that if we have an existing DNS record, the ansible rax_dns module can update the records to the new IP address of a newly created cloud server.

Ansible task to add dynamic instance to dynamic group inventory

First we need to have ansible add the new cloud server to an ansible group that we will be using for future tasks in our playbook:

- name: Add new cloud server to host group
      local_action:
        module: add_host
        hostname: "{{ item.name }}"
        ansible_ssh_host: "{{ item.rax_accessipv4 }}"
        ansible_ssh_user: root
        groupname: deploy
      with_items: rax.instances

What this tasks is doing is looking at the variable we registered with ansible “rax” and adding it to ansible’s internal list of groups and hosts within the group through ansible’s add_host module. The group we’re adding this newly created cloud server to is named “deploy”. We want to specify the hostname, IP address, and the username of the new server when adding the host to the ansible group.

Ansible task to create a Rackspace Cloud DNS A record

Next, we’ll create a task to add a forward DNS A record in Rackspace Cloud DNS for our new server:

- name: DNS - Create A record
      local_action:
        module: rax_dns_record
        credentials: "{{ credentials }}"
        domain: "{{ domain }}"
        name: "{{ name }}"
        data: "{{ item.rax_accessipv4 }}"
        type: A
      with_items: rax.instances
      register: a_record

Again we’re using the registered “rax” variable which contains our newly created instance. We’ve also added a new variable “{{ domain }}” that we’ll specify in the “vars” section of our playbook. The domain variable specifies the domain name we want to ensure exists, or create it, in Rackspace Cloud DNS.

Add domain to the list of vars at the top of the playbook:

- domain: enhancedtest.net

But wait! What if the domain does not yet exist in Rackspace Cloud DNS? The DNS A record addition will fail!

Ansible task to create a new domain in Rackspace Cloud DNS

Let’s make a new task to create a new domain if it does not yet exist in Rackspace Cloud DNS. This tasks will come before the task to add any DNS records.

- name: DNS - Domain create request
      local_action:
        module: rax_dns
        credentials: "{{ credentials }}"
        name: "{{ domain }}"
        email: "{{ dns_email }}"
      register: rax_dns

Rackspace Cloud DNS requires an email address to use as the admin contact for a domain’s DNS. Let’s add a new variable “dns_email” in our “vars” list at the top of our playbook. Note: the email address doesn’t actually have to exist or work, we just need to specify one to create the new domain.

Add dns_email to the list of vars at the top of the playbook:

- dns_email: admin@enhancedtest.net

Ansible task to create a new Rackspace Cloud PTR record

Now that we have created the new DNS domain and A record for our server, let’s create a DNS PTR record aka reverse DNS for the new server.

- name: DNS - Create PTR record
      local_action:
        module: rax_dns_record
        credentials: "{{ credentials }}"
        server: "{{ item.id }}"
        name: "{{ name }}"
        region: "{{ region }}"
        data: "{{ item.rax_accessipv4 }}"
        type: PTR
      with_items: rax.instances
      register: ptr_record

Ansible playbook to create new Rackspace Cloud Server and add DNS entries

Here’s what the playbook looks like now with our Rackspace Cloud DNS additions to create a new domain, add an A record for the new server, and add a PTR record.

---
- name: Create a Rackspace Cloud Server
  hosts: localhost
  user: root
  connection: local
  gather_facts: False

  vars:
   # Rackspace Cloud DNS settings:
   # domain - the domain we will be using for the new server
   - domain: enhancedtest.net
   # dns_email - admin email address for the new domain name
   - dns_email: admin@enhancedtest.net
   # this is the name we will see in the Rackspace Cloud control panel, and
   # this will also be the hostname of our new server
   - name: admin.enhancedtest.net
   # the flavor specifies the server side of our instance
   - flavor: performance1-1
   # the image specifies the linux distro we will use for our server
   # note: this image UUID is for Ubuntu 14.10 PVHVM
   - image: 0766e5df-d60a-4100-ae8c-07f27ec0148f
   # the region is the Rackspace Cloud region we want to build our server in
   - region: DFW
   # credentials specifies the location of our pyrax configuration file we created earlier
   - credentials: /Users/nick/.rackspace_cloud_credentials
   # I like to drop in my SSH pub key automatically when I create the server
   # so that I can ssh in without a password
   # Note: Instead of dropping in a file, you can use a stored Rackspace key
   # when you build the server by editing key_name below to your key's name.
   - files:
        /root/.ssh/authorized_keys: /Users/nick/.ssh/id_rsa.pub

  tasks:
    - name: Rackspace cloud server build request
      local_action:
        module: rax
        credentials: "{{ credentials }}"
        name: "{{ name }}"
        flavor: "{{ flavor }}"
        image: "{{ image }}"
        region: "{{ region }}"
        # key_name - specifies the Rackspace cloud key to add to the server upon creation
        #key_name: my_rackspace_key
        files: "{{ files }}"
        # wait - specifies that we should wait until the server is fully created before proceeding
        wait: yes
        # state - present means we want our server to exist
        state: present
        # specify that we want both a public network (public IPv4) and
        # a private network (10. aka service net)
        networks:
          - private
          - public
        # group - specifies metadata to add to the new server with a server group
        #group: deploy
      # register is an ansible term to save the output in to a variable named rax
      register: rax

    - name: Add new cloud server to host group
      local_action:
        module: add_host
        hostname: "{{ item.name }}"
        ansible_ssh_host: "{{ item.rax_accessipv4 }}"
        ansible_ssh_user: root
        groupname: deploy
      with_items: rax.instances

    - name: Add new instance to host group
      local_action:
        module: add_host
        hostname: "{{ item.name }}"
        ansible_ssh_host: "{{ item.rax_accessipv4 }}"
        ansible_ssh_user: root
        groupname: deploy
      with_items: rax.instances

    - name: DNS - Domain create request
      local_action:
        module: rax_dns
        credentials: "{{ credentials }}"
        name: "{{ domain }}"
        email: "{{ dns_email }}"
      register: rax_dns

    - name: DNS - Create A record
      local_action:
        module: rax_dns_record
        credentials: "{{ credentials }}"
        domain: "{{ domain }}"
        name: "{{ name }}"
        data: "{{ item.rax_accessipv4 }}"
        type: A
      with_items: rax.instances
      register: a_record

    - name: DNS - Create PTR record
      local_action:
        module: rax_dns_record
        credentials: "{{ credentials }}"
        server: "{{ item.id }}"
        name: "{{ name }}"
        region: "{{ region }}"
        data: "{{ item.rax_accessipv4 }}"
        type: PTR
      with_items: rax.instances
      register: ptr_record

Let’s give our updated playbook a test run and see what happens!

Run the playbook the same way as before. Note: I’m not including -vvvv this time as the output can be extremely verbose, but I always run it when testing, debugging and troubleshooting.

ansible-playbook -i virtualenv-inventory.yml build-cloud-server.yml

Output from the playbook: https://gist.github.com/nicholaskuechler/f6a01223252b89dea959

Success! Some interesting things to note:

• The server already existed with the name we specified in the playbook, so a new cloud server was not created
• The new DNS domain was added successfully
• The new A record was added successfully
• The new PTR record was added successfully

What happens if we run the playbook again? Will it create more DNS records again? Let’s give it a try. Here’s the output from my workstation: https://gist.github.com/nicholaskuechler/d2562ef1c780cdeffe14

Since the domain, the A record, and PTR record already existed in DNS, the tasks were marked as OK and no changes were made. If changes were made, the tasks would be denoted with “changed:” rather than “ok:” in the task output. You can also see the overall playbook tasks ok / changed / failed status in the play recap summary.

Use ansible to install base packages on a Rackspace Cloud server

Now that we have successfully created a new cloud server and performed routine system administrator tasks like setting up DNS entries, let’s go ahead and install our favorite packages on our new server. On every server I use, I want vim, git and screen to be available, so let’s make sure those packages are definitely installed.

There are two different ways we can accomplish the package installations:

1.) Add a new task to install a list of packages

2.) Add a new ansible role that installs base packages that we can reuse in other playbooks

I prefer option #2 which allows me to re-use my code in other playbooks. I install the same set of base packages on each new server I create, so this will definitely come in handy for me in the future.

But let’s cover both options!

Add a new ansible task to install a list of packages

Here’s how we can add a new task to our existing ansible playbook to run apt update and install a bunch of our favorite packages on our new Rackspace cloud server. The task basically loops through a list of packages specified by with_items and uses apt to install the individual item, which is the package.

- name: Install Packages
  hosts: deploy
  user: root
  gather_facts: True

  tasks:
  - name: Run apt update
    apt: update_cache=yes

  - name: install packages
    action: apt state=installed pkg={{ item }}
    with_items:
    - vim
    - git
    - screen
    tags:
    - packages

Important note: Since we’re using ubuntu as our linux distro, we need to use the apt ansible module to install packages. If we were using CentOS or Fedora we would change this to yum.

Note: The ansible hosts group we’re using is “deploy” which we specified previously when adding the newly created cloud server to an internal, dynamic ansible host group. The important piece here is to make sure both group names match if you decide to use a different name!

Create a new ansible role to install a list of base packages

Using ansible roles to install our favorite packages is a slightly more complicated method than adding the install packages task to the existing playbook, but it is much more reusable and using roles is generally the preferred ansible way.

Make ansible role directories

First, we need to create a directory structure for our ansible roles. All ansible roles will live in a subdirectory of the directory your playbook lives in called “roles”. Let’s call our new role “base”

In our example, our playbook build-cloud-server.yml lives in the directory ansible-rackspace-servers, so in the ansible-rackspace-servers directory let’s make a new directory named “roles”

mkdir roles

Then make the roles directory for our “base” role, and a couple standard directories roles in ansible use:

cd roles
mkdir -p base/{files,handlers,meta,templates,tasks,vars}

Note: We’re not going to be using all of these standard role directories like meta and vars at this time, but we’ll go ahead and create them now in case we expand our role in the future.

Create package installation task in the new base role

With our directory structure set up for our new “base” role, we need to create a task in this role to install our favorite packages.

Create a new file in roles/base/tasks/ directory named main.yml. By default, all ansible roles will have a main.yml in the role’s tasks subdirectory.

Inside of main.yml, let’s write a simple task to install our packages. It’s going to look very similar to the method of adding the package install tasks directly in the playbook, but we won’t need to specify hosts, users, etc.

---
- name: Run apt update
  apt: update_cache=yes

- name: Install apt packages
  apt: pkg={{ item }} state=installed
  with_items:
    - vim
    - git
    - screen
  tags:
    - packages

That’s it! We’ve just created a new role named “base” to install our favorite packages. The best part about this is that we can expand our base role to perform other tasks that need to run on all of our servers, such as adding an admin user or setting up ntp.

Using the newly created base role in our build cloud server playbook

With our base role created, we now need to modify our playbook to use the role to install our base configuration and packages.

In our playbook, we need to add:

- name: Install base packages to new cloud server
  hosts: deploy
  user: root
  gather_facts: True
  roles:
    - base

The “roles” section specifies which roles to run on the hosts specified in the dynamic “deploy” group. For example, if we had an nginx ansible role we wanted to use, we could easily add it to the list of roles to use, like this:

roles:
    - base
    - nginx

We can also easily use the new “base” role we just created in other ansible playbooks.

Run the playbook with the new base role to install packages

Let’s run our playbook now using the new base role to install packages on our new cloud server. Here’s the ansible-playbook output from my test run: https://gist.github.com/nicholaskuechler/72ae99ef1acc85a712fe

Success! The “changed” status under “install apt packages” denotes our 3 favorite packages were installed, and the recap tells us there were no failures.

Final version of playbook to create a Rackspace Cloud Server and install packages

Here’s the final version of our playbook:

---
- name: Create a Rackspace Cloud Server
  hosts: localhost
  user: root
  connection: local
  gather_facts: False

  vars:
   # Rackspace Cloud DNS settings:
   # domain - the domain we will be using for the new server
   - domain: enhancedtest.net
   # dns_email - admin email address for the new domain name
   - dns_email: admin@enhancedtest.net
   # this is the name we will see in the Rackspace Cloud control panel, and
   # this will also be the hostname of our new server
   - name: admin.enhancedtest.net
   # the flavor specifies the server side of our instance
   - flavor: performance1-1
   # the image specifies the linux distro we will use for our server
   - image: 0766e5df-d60a-4100-ae8c-07f27ec0148f
   # the region is the Rackspace Cloud region we want to build our server in
   - region: DFW
   # credentials specifies the location of our pyrax configuration file we created earlier
   - credentials: /Users/nick/.rackspace_cloud_credentials
   # I like to drop in my SSH pub key automatically when I create the server
   # so that I can ssh in without a password
   # Note: Instead of dropping in a file, you can use a stored Rackspace key
   # when you build the server by editing key_name below to your key's name.
   - files:
        /root/.ssh/authorized_keys: /Users/nick/.ssh/id_rsa.pub

  tasks:
    - name: Rackspace cloud server build request
      local_action:
        module: rax
        credentials: "{{ credentials }}"
        name: "{{ name }}"
        flavor: "{{ flavor }}"
        image: "{{ image }}"
        region: "{{ region }}"
        # key_name - specifies the Rackspace cloud key to add to the server upon creation
        #key_name: my_rackspace_key
        files: "{{ files }}"
        # wait - specifies that we should wait until the server is fully created before proceeding
        wait: yes
        # state - present means we want our server to exist
        state: present
        # specify that we want both a public network (public IPv4) and
        # a private network (10. aka service net)
        networks:
          - private
          - public
        # group - specifies metadata to add to the new server with a server group
        #group: deploy
      # register is an ansible term to save the output in to a variable named rax
      register: rax

    - name: Add new cloud server to host group
      local_action:
        module: add_host
        hostname: "{{ item.name }}"
        ansible_ssh_host: "{{ item.rax_accessipv4 }}"
        ansible_ssh_user: root
        groupname: deploy
      with_items: rax.instances

    - name: Add new instance to host group
      local_action:
        module: add_host
        hostname: "{{ item.name }}"
        ansible_ssh_host: "{{ item.rax_accessipv4 }}"
        ansible_ssh_user: root
        groupname: deploy
      with_items: rax.instances

    - name: DNS - Domain create request
      local_action:
        module: rax_dns
        credentials: "{{ credentials }}"
        name: "{{ domain }}"
        email: "{{ dns_email }}"
      register: rax_dns

    - name: DNS - Create A record
      local_action:
        module: rax_dns_record
        credentials: "{{ credentials }}"
        domain: "{{ domain }}"
        name: "{{ name }}"
        data: "{{ item.rax_accessipv4 }}"
        type: A
      with_items: rax.instances
      register: a_record

    - name: DNS - Create PTR record
      local_action:
        module: rax_dns_record
        credentials: "{{ credentials }}"
        server: "{{ item.id }}"
        name: "{{ name }}"
        region: "{{ region }}"
        data: "{{ item.rax_accessipv4 }}"
        type: PTR
      with_items: rax.instances
      register: ptr_record

- name: Install base packages to new cloud server
  hosts: deploy
  user: root
  gather_facts: True
  roles:
    - base

Source for ansible-rackspace-servers-example on GitHub

You can find the complete source for this ansible Rackspace cloud servers example on my github: ansible-rackspace-servers-example

See also

• Jekyll with Clean URLs Hosted at Rackspace Cloud Files
• Auto-scale Rackspace Cloud servers with Fabric and Celery
• Trading Tools – Stock Price Decline Checker
• Pelican, a static site generator written in Python